Second approach to inform driver about metadata. Let user decide if
metadata should be supported or not.

Add this flag to allow user to inform driver that metadata is used.
Set flag is sent to driver via exsisting ndo_bpf call in flag field.

Signed-off-by: Michal Swiatkowski <[email protected]>

Definition is only a proposal. There should be free place for 8B of tx
timestamp.

Signed-off-by: Michal Swiatkowski <[email protected]>

As starting point add vlan id and rss hash if xdp metadata is supported.

Add xd_metadata_support field in VSI to allow easy passing this value to
ring configuration.

Signed-off-by: Michal Swiatkowski <[email protected]>

Variable "err" is initialised to -EINVAL so that this error code is
returned when something goes wrong in libbpf_find_prog_btf_id().
However, a recent change in the function made use of the variable in
such a way that it is set to 0 if retrieving linear information on the
program is successful, and this 0 value remains if we error out on
failures at later stages.

Let's fix this by setting err to -EINVAL later in the function.

Fixes: e9fc3ce ("libbpf: Streamline error reporting for high-level APIs")
Signed-off-by: Quentin Monnet <[email protected]>
Signed-off-by: Andrii Nakryiko <[email protected]>
Link: https://lore.kernel.org/bpf/[email protected]

As part of the effort to move towards a v1.0 for libbpf, rename
btf__load() function, used to "upload" BTF information into the kernel,
as btf__load_into_kernel(). This new name better reflects what the
function does.

References:

- libbpf/libbpf#278
- https://github.com/libbpf/libbpf/wiki/Libbpf:-the-road-to-v1.0#btfh-apis

Signed-off-by: Quentin Monnet <[email protected]>
Signed-off-by: Andrii Nakryiko <[email protected]>
Acked-by: John Fastabend <[email protected]>
Link: https://lore.kernel.org/bpf/[email protected]

Rename function btf__get_from_id() as btf__load_from_kernel_by_id() to
better indicate what the function does. Change the new function so that,
instead of requiring a pointer to the pointer to update and returning
with an error code, it takes a single argument (the id of the BTF
object) and returns the corresponding pointer. This is more in line with
the existing constructors.

The other tools calling the (soon-to-be) deprecated btf__get_from_id()
function will be updated in a future commit.

References:

- libbpf/libbpf#278
- https://github.com/libbpf/libbpf/wiki/Libbpf:-the-road-to-v1.0#btfh-apis

Signed-off-by: Quentin Monnet <[email protected]>
Signed-off-by: Andrii Nakryiko <[email protected]>
Acked-by: John Fastabend <[email protected]>
Link: https://lore.kernel.org/bpf/[email protected]

Make sure to call btf__free() (and not simply free(), which does not
free all pointers stored in the struct) on pointers to struct btf
objects retrieved at various locations.

These were found while updating the calls to btf__get_from_id().

Fixes: 999d82c ("tools/bpf: enhance test_btf file testing to test func info")
Fixes: 254471e ("tools/bpf: bpftool: add support for func types")
Fixes: 7b612e2 ("perf tools: Synthesize PERF_RECORD_* for loaded BPF programs")
Fixes: d56354d ("perf tools: Save bpf_prog_info and BTF of new BPF programs")
Fixes: 47c09d6 ("bpftool: Introduce "prog profile" command")
Fixes: fa853c4 ("perf stat: Enable counting events for BPF programs")
Signed-off-by: Quentin Monnet <[email protected]>
Signed-off-by: Andrii Nakryiko <[email protected]>
Link: https://lore.kernel.org/bpf/[email protected]

Replace the calls to function btf__get_from_id(), which we plan to
deprecate before the library reaches v1.0, with calls to
btf__load_from_kernel_by_id() in tools/ (bpftool, perf, selftests).
Update the surrounding code accordingly (instead of passing a pointer to
the btf struct, get it as a return value from the function).

Signed-off-by: Quentin Monnet <[email protected]>
Signed-off-by: Andrii Nakryiko <[email protected]>
Acked-by: John Fastabend <[email protected]>
Link: https://lore.kernel.org/bpf/[email protected]

Add a new API function btf__load_from_kernel_by_id_split(), which takes
a pointer to a base BTF object in order to support split BTF objects
when retrieving BTF information from the kernel.

Reference: libbpf/libbpf#314

Signed-off-by: Quentin Monnet <[email protected]>
Signed-off-by: Andrii Nakryiko <[email protected]>
Acked-by: John Fastabend <[email protected]>
Link: https://lore.kernel.org/bpf/[email protected]

Currently, the igc driver supports timestamping only one Tx packet at a
time. During the transmission flow, the skb that requires hardware
timestamping is saved in adapter->ptp_tx_skb. Once hardware has the
timestamp, an interrupt is delivered, and adapter->ptp_tx_work is
scheduled. In igc_ptp_tx_work(), we read the timestamp register, update
adapter->ptp_tx_skb, and notify the network stack.

While the thread executing the transmission flow (the user process
running in kernel mode) and the thread executing ptp_tx_work don't
access adapter->ptp_tx_skb concurrently, there are two other places
where adapter->ptp_tx_skb is accessed: igc_ptp_tx_hang() and
igc_ptp_suspend().

igc_ptp_tx_hang() is executed by the adapter->watchdog_task worker
thread which runs periodically so it is possible we have two threads
accessing ptp_tx_skb at the same time. Consider the following scenario:
right after __IGC_PTP_TX_IN_PROGRESS is set in igc_xmit_frame_ring(),
igc_ptp_tx_hang() is executed. Since adapter->ptp_tx_start hasn't been
written yet, this is considered a timeout and adapter->ptp_tx_skb is
cleaned up.

This patch fixes the issue described above by adding the ptp_tx_lock to
protect access to ptp_tx_skb and ptp_tx_start fields from igc_adapter.
Since igc_xmit_frame_ring() called in atomic context by the networking
stack, ptp_tx_lock is defined as a spinlock.

With the introduction of the ptp_tx_lock, the __IGC_PTP_TX_IN_PROGRESS
flag doesn't provide much of a use anymore so this patch gets rid of it.

Signed-off-by: Andre Guedes <[email protected]>

Handling of TX timestamp interrupt should be simple enough to not cause
issues during the interrupt context. This way, the processing is
simplified and potentially more performant.

This patch is inspired by the i40 driver approach.

Signed-off-by: Vinicius Costa Gomes <[email protected]>

Adds support for using the four sets of timestamping registers that
i225 has available for TX.

In some TSN workloads, where multiple applications request hardware
transmission timestamps, it was possible that some of those requests
were denied because the only in use register was already occupied.

Signed-off-by: Vinicius Costa Gomes <[email protected]>

Now that the timestamping is done in interrupt context we should
protect against concurrent access using irq safe locks.

Signed-off-by: Vinicius Costa Gomes <[email protected]>

New flag used by bpf programs or AF_XDP applications to inform the
driver to use XDP metadata information.

Signed-off-by: Ederson de Souza <[email protected]>

New XDP_FLAGS_USE_METADATA should be available for AF_XDP applications,
which use link XDP.

Signed-off-by: Ederson de Souza <[email protected]>

A network driver may need to get the BTF ID information currently
associated with xdp_meta_generic. To do so, it needs to access
btf_obj_id and bpf_get_btf_vmlinux functions, so, let's export them.

Signed-off-by: Ederson de Souza <[email protected]>

A network driver interested in using its associated BTF ID needs to find
its associated BTF. This patch introduces a new function,
btf_get_from_module that allows a module to get the BTF associated with
it.

Signed-off-by: Ederson de Souza <[email protected]>

Using XDP hints, driver adds the PTP timestamp of when a packet was
received by the i225 NIC.

Signed-off-by: Ederson de Souza <[email protected]>

ADD the PTP timestamp of when a packet was transmitted to the XDP hints.
An application using AF_XDP can get this timestamp by inspecting the XDP
frame metadata when it gets to the completion queue.

One notable difference from TX timestamp for SKB, is that the XDP frame
actually resides in the UMEM. As such, the timestamp is added to the
frame, and user space applications can access it when the frame is sent
to the completion queue.

When performing the clean-up of TX descriptors, driver will check if an
XDP socket frame is "expecting" a TX timestamp. If so, driver will stop
clean-up to give an opportunity for the TX timestamp interrupt arrive.

Signed-off-by: Ederson de Souza <[email protected]>

This information can be used by user space applications to determine how
much headroom is needed for the XDP frame.

igc driver is also changed to add this new information.

Signed-off-by: Ederson de Souza <[email protected]>

Two new pairs of helpers: `xsk_umem__adjust_prod_data` and
`xsk_umem__adjust_prod_data_meta` for data that is being produced by the
application - such as data that will be sent; and
`xsk_umem__adjust_cons_data` and `xsk_umem__adjust_cons_data_meta`,
for data being consumed - such as data obtained from the completion
queue.

Those function should usually be used on data obtained via
`xsk_umem__get_data`. Didn't change this function to avoid API breaks.

Signed-off-by: Ederson de Souza <[email protected]>

A new set of functions to help get the BTF definition of XDP hints
structure and get the information based on it.

`xsk_umem__btf_id` helps retrieve the BTF id of XDP metadata.
`xsk_btf__init` sets up a context based on the BTF, including a hashmap,
so that subsequent queries are faster.
`xsk_btf__read` returns a pointer to the position in the XDP metadata
containing a given field.
`xsk_btf__has_field` checks the presence of a field in the BTF.
`xsk_btf__free` frees up the context.

Besides those, a macro `XSK_BTF_READ_INTO` acts as a convenient helper
to read the field contents into a given variable.

Note that currently, the hashmap used to speed-up offset location into
the BTF doesn't use the field name as a string as key to the hashmap. It
directly uses the pointer value instead, as it is expected that most of
time, field names will be addressed by a shared constant string residing
on read-only memory, thus saving some time. If this assumption is not
entirely true, this optimisation needs to be rethought (or discarded
altogether).

Signed-off-by: Ederson de Souza <[email protected]>

Using -D option, xdpsock now shows the RX or TX timestamp of last
sent/received packets (for rx only or tx only modes).

Signed-off-by: Ederson de Souza <[email protected]>

Assuming that XDP metadata related to RX does not overlap with metadata
related to TX, this patch splits TX and RX data inside xdp_meta_generic
into different anonymous structs united by another anonymous union.

This allows more data to fit in the 32 bytes target size for
xdp_meta_generic. With help of BTF CO-RE, this is transparent for BPF
applications. For AF_XDP ones, libbpf has been modified to account for
the new layout.

The flip side is that holes appear in the struct, and one has to
carefully take care of padding, so that `btf_id` field is still at the
very end of struct.

Signed-off-by: Ederson de Souza <[email protected]>