Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

scheduler: (un)mark a subflow as "stale" #349

Open
Tracked by #350
matttbe opened this issue Feb 1, 2023 · 3 comments
Open
Tracked by #350

scheduler: (un)mark a subflow as "stale" #349

matttbe opened this issue Feb 1, 2023 · 3 comments
Assignees
Labels
enhancement sched packets scheduler

Comments

@matttbe
Copy link
Member

matttbe commented Feb 1, 2023

The core is currently marking a subflow as "stale" after a certain number of losses (net.mptcp.stale_loss_cnt).

A packet scheduler might be able to do the same for other reasons: e.g. if a subflow is considered as "really bad" by the scheduler (e.g. too high latency, too unstable, etc.) and it should not be used for the moment, the scheduler will want to mark the subflow as "stale" and have a regular probing (linked to #348).

The packet scheduler should either:

  • be able to replace what the core is doing
  • or managing a new bit field

Because the "stale" bit is tightly linked to the RTO (a subflow will no longer be considered as "stale" by the core and unmarked when a packet has been acked on this subflow), it should because to manage a new bit where only the packet scheduler will be able to mark and unmark it as "temporarily unusable" ("skip this subflow if possible").

@sferlin
Copy link

sferlin commented May 3, 2023

It sounds a bit strange that putting a subflow to "stale" state can be defined from outside the scheduler. Can you provide a bit of context here about why this decision (the sysctl above) was defined like it is?
In principle, if the scheduler has the logic of putting a subflow into "stale" state, I would suggest that it can precede what core is doing. Not all schedulers will include this logic, to "discard" (even temporarily) a subflow from the scheduling decision.

@pabeni
Copy link

pabeni commented May 25, 2023

It sounds a bit strange that putting a subflow to "stale" state can be defined from outside the scheduler.

To hopefully clarify: currently subflows are marked (un unmarked) as stale by the build-in scheduler. The main point behind this issue is that currently the MPTCP core does not offer clear APIs to the eBPF scheduler to do such action.

We need to clearly define such APIs, re-factor the core scheduler to used them instead of the current direct suflow struct manipulation and write some eBPF code to leverage such new APIs.

@geliangtang geliangtang self-assigned this Jun 28, 2023
@matttbe
Copy link
Member Author

matttbe commented Jul 30, 2023

Following the discussion we had with Mat on the ML, it sounds better for a packet scheduler not to modify the "stale" bit but use a new one, see the new note in the issue description:

Because the "stale" bit is tightly linked to the RTO (a subflow will no longer be considered as "stale" by the core and unmarked when a packet has been acked on this subflow), it should because to manage a new bit where only the packet scheduler will be able to mark and unmark it as "temporarily unusable" ("skip this subflow if possible").

jenkins-tessares pushed a commit that referenced this issue Nov 17, 2023
Enable the cpu v4 tests for LoongArch. Currently, we don't have BPF
trampoline in LoongArch JIT, so the fentry test `test_ptr_struct_arg`
still failed, will followup.

Test result attached below:

  # ./test_progs -t verifier_sdiv,verifier_movsx,verifier_ldsx,verifier_gotol,verifier_bswap
  #316/1   verifier_bswap/BSWAP, 16:OK
  #316/2   verifier_bswap/BSWAP, 16 @unpriv:OK
  #316/3   verifier_bswap/BSWAP, 32:OK
  #316/4   verifier_bswap/BSWAP, 32 @unpriv:OK
  #316/5   verifier_bswap/BSWAP, 64:OK
  #316/6   verifier_bswap/BSWAP, 64 @unpriv:OK
  #316     verifier_bswap:OK
  #330/1   verifier_gotol/gotol, small_imm:OK
  #330/2   verifier_gotol/gotol, small_imm @unpriv:OK
  #330     verifier_gotol:OK
  #338/1   verifier_ldsx/LDSX, S8:OK
  #338/2   verifier_ldsx/LDSX, S8 @unpriv:OK
  #338/3   verifier_ldsx/LDSX, S16:OK
  #338/4   verifier_ldsx/LDSX, S16 @unpriv:OK
  #338/5   verifier_ldsx/LDSX, S32:OK
  #338/6   verifier_ldsx/LDSX, S32 @unpriv:OK
  #338/7   verifier_ldsx/LDSX, S8 range checking, privileged:OK
  #338/8   verifier_ldsx/LDSX, S16 range checking:OK
  #338/9   verifier_ldsx/LDSX, S16 range checking @unpriv:OK
  #338/10  verifier_ldsx/LDSX, S32 range checking:OK
  #338/11  verifier_ldsx/LDSX, S32 range checking @unpriv:OK
  #338     verifier_ldsx:OK
  #349/1   verifier_movsx/MOV32SX, S8:OK
  #349/2   verifier_movsx/MOV32SX, S8 @unpriv:OK
  #349/3   verifier_movsx/MOV32SX, S16:OK
  #349/4   verifier_movsx/MOV32SX, S16 @unpriv:OK
  #349/5   verifier_movsx/MOV64SX, S8:OK
  #349/6   verifier_movsx/MOV64SX, S8 @unpriv:OK
  #349/7   verifier_movsx/MOV64SX, S16:OK
  #349/8   verifier_movsx/MOV64SX, S16 @unpriv:OK
  #349/9   verifier_movsx/MOV64SX, S32:OK
  #349/10  verifier_movsx/MOV64SX, S32 @unpriv:OK
  #349/11  verifier_movsx/MOV32SX, S8, range_check:OK
  #349/12  verifier_movsx/MOV32SX, S8, range_check @unpriv:OK
  #349/13  verifier_movsx/MOV32SX, S16, range_check:OK
  #349/14  verifier_movsx/MOV32SX, S16, range_check @unpriv:OK
  #349/15  verifier_movsx/MOV32SX, S16, range_check 2:OK
  #349/16  verifier_movsx/MOV32SX, S16, range_check 2 @unpriv:OK
  #349/17  verifier_movsx/MOV64SX, S8, range_check:OK
  #349/18  verifier_movsx/MOV64SX, S8, range_check @unpriv:OK
  #349/19  verifier_movsx/MOV64SX, S16, range_check:OK
  #349/20  verifier_movsx/MOV64SX, S16, range_check @unpriv:OK
  #349/21  verifier_movsx/MOV64SX, S32, range_check:OK
  #349/22  verifier_movsx/MOV64SX, S32, range_check @unpriv:OK
  #349/23  verifier_movsx/MOV64SX, S16, R10 Sign Extension:OK
  #349/24  verifier_movsx/MOV64SX, S16, R10 Sign Extension @unpriv:OK
  #349     verifier_movsx:OK
  #361/1   verifier_sdiv/SDIV32, non-zero imm divisor, check 1:OK
  #361/2   verifier_sdiv/SDIV32, non-zero imm divisor, check 1 @unpriv:OK
  #361/3   verifier_sdiv/SDIV32, non-zero imm divisor, check 2:OK
  #361/4   verifier_sdiv/SDIV32, non-zero imm divisor, check 2 @unpriv:OK
  #361/5   verifier_sdiv/SDIV32, non-zero imm divisor, check 3:OK
  #361/6   verifier_sdiv/SDIV32, non-zero imm divisor, check 3 @unpriv:OK
  #361/7   verifier_sdiv/SDIV32, non-zero imm divisor, check 4:OK
  #361/8   verifier_sdiv/SDIV32, non-zero imm divisor, check 4 @unpriv:OK
  #361/9   verifier_sdiv/SDIV32, non-zero imm divisor, check 5:OK
  #361/10  verifier_sdiv/SDIV32, non-zero imm divisor, check 5 @unpriv:OK
  #361/11  verifier_sdiv/SDIV32, non-zero imm divisor, check 6:OK
  #361/12  verifier_sdiv/SDIV32, non-zero imm divisor, check 6 @unpriv:OK
  #361/13  verifier_sdiv/SDIV32, non-zero imm divisor, check 7:OK
  #361/14  verifier_sdiv/SDIV32, non-zero imm divisor, check 7 @unpriv:OK
  #361/15  verifier_sdiv/SDIV32, non-zero imm divisor, check 8:OK
  #361/16  verifier_sdiv/SDIV32, non-zero imm divisor, check 8 @unpriv:OK
  #361/17  verifier_sdiv/SDIV32, non-zero reg divisor, check 1:OK
  #361/18  verifier_sdiv/SDIV32, non-zero reg divisor, check 1 @unpriv:OK
  #361/19  verifier_sdiv/SDIV32, non-zero reg divisor, check 2:OK
  #361/20  verifier_sdiv/SDIV32, non-zero reg divisor, check 2 @unpriv:OK
  #361/21  verifier_sdiv/SDIV32, non-zero reg divisor, check 3:OK
  #361/22  verifier_sdiv/SDIV32, non-zero reg divisor, check 3 @unpriv:OK
  #361/23  verifier_sdiv/SDIV32, non-zero reg divisor, check 4:OK
  #361/24  verifier_sdiv/SDIV32, non-zero reg divisor, check 4 @unpriv:OK
  #361/25  verifier_sdiv/SDIV32, non-zero reg divisor, check 5:OK
  #361/26  verifier_sdiv/SDIV32, non-zero reg divisor, check 5 @unpriv:OK
  #361/27  verifier_sdiv/SDIV32, non-zero reg divisor, check 6:OK
  #361/28  verifier_sdiv/SDIV32, non-zero reg divisor, check 6 @unpriv:OK
  #361/29  verifier_sdiv/SDIV32, non-zero reg divisor, check 7:OK
  #361/30  verifier_sdiv/SDIV32, non-zero reg divisor, check 7 @unpriv:OK
  #361/31  verifier_sdiv/SDIV32, non-zero reg divisor, check 8:OK
  #361/32  verifier_sdiv/SDIV32, non-zero reg divisor, check 8 @unpriv:OK
  #361/33  verifier_sdiv/SDIV64, non-zero imm divisor, check 1:OK
  #361/34  verifier_sdiv/SDIV64, non-zero imm divisor, check 1 @unpriv:OK
  #361/35  verifier_sdiv/SDIV64, non-zero imm divisor, check 2:OK
  #361/36  verifier_sdiv/SDIV64, non-zero imm divisor, check 2 @unpriv:OK
  #361/37  verifier_sdiv/SDIV64, non-zero imm divisor, check 3:OK
  #361/38  verifier_sdiv/SDIV64, non-zero imm divisor, check 3 @unpriv:OK
  #361/39  verifier_sdiv/SDIV64, non-zero imm divisor, check 4:OK
  #361/40  verifier_sdiv/SDIV64, non-zero imm divisor, check 4 @unpriv:OK
  #361/41  verifier_sdiv/SDIV64, non-zero imm divisor, check 5:OK
  #361/42  verifier_sdiv/SDIV64, non-zero imm divisor, check 5 @unpriv:OK
  #361/43  verifier_sdiv/SDIV64, non-zero imm divisor, check 6:OK
  #361/44  verifier_sdiv/SDIV64, non-zero imm divisor, check 6 @unpriv:OK
  #361/45  verifier_sdiv/SDIV64, non-zero reg divisor, check 1:OK
  #361/46  verifier_sdiv/SDIV64, non-zero reg divisor, check 1 @unpriv:OK
  #361/47  verifier_sdiv/SDIV64, non-zero reg divisor, check 2:OK
  #361/48  verifier_sdiv/SDIV64, non-zero reg divisor, check 2 @unpriv:OK
  #361/49  verifier_sdiv/SDIV64, non-zero reg divisor, check 3:OK
  #361/50  verifier_sdiv/SDIV64, non-zero reg divisor, check 3 @unpriv:OK
  #361/51  verifier_sdiv/SDIV64, non-zero reg divisor, check 4:OK
  #361/52  verifier_sdiv/SDIV64, non-zero reg divisor, check 4 @unpriv:OK
  #361/53  verifier_sdiv/SDIV64, non-zero reg divisor, check 5:OK
  #361/54  verifier_sdiv/SDIV64, non-zero reg divisor, check 5 @unpriv:OK
  #361/55  verifier_sdiv/SDIV64, non-zero reg divisor, check 6:OK
  #361/56  verifier_sdiv/SDIV64, non-zero reg divisor, check 6 @unpriv:OK
  #361/57  verifier_sdiv/SMOD32, non-zero imm divisor, check 1:OK
  #361/58  verifier_sdiv/SMOD32, non-zero imm divisor, check 1 @unpriv:OK
  #361/59  verifier_sdiv/SMOD32, non-zero imm divisor, check 2:OK
  #361/60  verifier_sdiv/SMOD32, non-zero imm divisor, check 2 @unpriv:OK
  #361/61  verifier_sdiv/SMOD32, non-zero imm divisor, check 3:OK
  #361/62  verifier_sdiv/SMOD32, non-zero imm divisor, check 3 @unpriv:OK
  #361/63  verifier_sdiv/SMOD32, non-zero imm divisor, check 4:OK
  #361/64  verifier_sdiv/SMOD32, non-zero imm divisor, check 4 @unpriv:OK
  #361/65  verifier_sdiv/SMOD32, non-zero imm divisor, check 5:OK
  #361/66  verifier_sdiv/SMOD32, non-zero imm divisor, check 5 @unpriv:OK
  #361/67  verifier_sdiv/SMOD32, non-zero imm divisor, check 6:OK
  #361/68  verifier_sdiv/SMOD32, non-zero imm divisor, check 6 @unpriv:OK
  #361/69  verifier_sdiv/SMOD32, non-zero reg divisor, check 1:OK
  #361/70  verifier_sdiv/SMOD32, non-zero reg divisor, check 1 @unpriv:OK
  #361/71  verifier_sdiv/SMOD32, non-zero reg divisor, check 2:OK
  #361/72  verifier_sdiv/SMOD32, non-zero reg divisor, check 2 @unpriv:OK
  #361/73  verifier_sdiv/SMOD32, non-zero reg divisor, check 3:OK
  #361/74  verifier_sdiv/SMOD32, non-zero reg divisor, check 3 @unpriv:OK
  #361/75  verifier_sdiv/SMOD32, non-zero reg divisor, check 4:OK
  #361/76  verifier_sdiv/SMOD32, non-zero reg divisor, check 4 @unpriv:OK
  #361/77  verifier_sdiv/SMOD32, non-zero reg divisor, check 5:OK
  #361/78  verifier_sdiv/SMOD32, non-zero reg divisor, check 5 @unpriv:OK
  #361/79  verifier_sdiv/SMOD32, non-zero reg divisor, check 6:OK
  #361/80  verifier_sdiv/SMOD32, non-zero reg divisor, check 6 @unpriv:OK
  #361/81  verifier_sdiv/SMOD64, non-zero imm divisor, check 1:OK
  #361/82  verifier_sdiv/SMOD64, non-zero imm divisor, check 1 @unpriv:OK
  #361/83  verifier_sdiv/SMOD64, non-zero imm divisor, check 2:OK
  #361/84  verifier_sdiv/SMOD64, non-zero imm divisor, check 2 @unpriv:OK
  #361/85  verifier_sdiv/SMOD64, non-zero imm divisor, check 3:OK
  #361/86  verifier_sdiv/SMOD64, non-zero imm divisor, check 3 @unpriv:OK
  #361/87  verifier_sdiv/SMOD64, non-zero imm divisor, check 4:OK
  #361/88  verifier_sdiv/SMOD64, non-zero imm divisor, check 4 @unpriv:OK
  #361/89  verifier_sdiv/SMOD64, non-zero imm divisor, check 5:OK
  #361/90  verifier_sdiv/SMOD64, non-zero imm divisor, check 5 @unpriv:OK
  #361/91  verifier_sdiv/SMOD64, non-zero imm divisor, check 6:OK
  #361/92  verifier_sdiv/SMOD64, non-zero imm divisor, check 6 @unpriv:OK
  #361/93  verifier_sdiv/SMOD64, non-zero imm divisor, check 7:OK
  #361/94  verifier_sdiv/SMOD64, non-zero imm divisor, check 7 @unpriv:OK
  #361/95  verifier_sdiv/SMOD64, non-zero imm divisor, check 8:OK
  #361/96  verifier_sdiv/SMOD64, non-zero imm divisor, check 8 @unpriv:OK
  #361/97  verifier_sdiv/SMOD64, non-zero reg divisor, check 1:OK
  #361/98  verifier_sdiv/SMOD64, non-zero reg divisor, check 1 @unpriv:OK
  #361/99  verifier_sdiv/SMOD64, non-zero reg divisor, check 2:OK
  #361/100 verifier_sdiv/SMOD64, non-zero reg divisor, check 2 @unpriv:OK
  #361/101 verifier_sdiv/SMOD64, non-zero reg divisor, check 3:OK
  #361/102 verifier_sdiv/SMOD64, non-zero reg divisor, check 3 @unpriv:OK
  #361/103 verifier_sdiv/SMOD64, non-zero reg divisor, check 4:OK
  #361/104 verifier_sdiv/SMOD64, non-zero reg divisor, check 4 @unpriv:OK
  #361/105 verifier_sdiv/SMOD64, non-zero reg divisor, check 5:OK
  #361/106 verifier_sdiv/SMOD64, non-zero reg divisor, check 5 @unpriv:OK
  #361/107 verifier_sdiv/SMOD64, non-zero reg divisor, check 6:OK
  #361/108 verifier_sdiv/SMOD64, non-zero reg divisor, check 6 @unpriv:OK
  #361/109 verifier_sdiv/SMOD64, non-zero reg divisor, check 7:OK
  #361/110 verifier_sdiv/SMOD64, non-zero reg divisor, check 7 @unpriv:OK
  #361/111 verifier_sdiv/SMOD64, non-zero reg divisor, check 8:OK
  #361/112 verifier_sdiv/SMOD64, non-zero reg divisor, check 8 @unpriv:OK
  #361/113 verifier_sdiv/SDIV32, zero divisor:OK
  #361/114 verifier_sdiv/SDIV32, zero divisor @unpriv:OK
  #361/115 verifier_sdiv/SDIV64, zero divisor:OK
  #361/116 verifier_sdiv/SDIV64, zero divisor @unpriv:OK
  #361/117 verifier_sdiv/SMOD32, zero divisor:OK
  #361/118 verifier_sdiv/SMOD32, zero divisor @unpriv:OK
  #361/119 verifier_sdiv/SMOD64, zero divisor:OK
  #361/120 verifier_sdiv/SMOD64, zero divisor @unpriv:OK
  #361     verifier_sdiv:OK
  Summary: 5/163 PASSED, 0 SKIPPED, 0 FAILED

  # ./test_progs -t ldsx_insn
  test_map_val_and_probed_memory:PASS:test_ldsx_insn__open 0 nsec
  test_map_val_and_probed_memory:PASS:test_ldsx_insn__load 0 nsec
  libbpf: prog 'test_ptr_struct_arg': failed to attach: ERROR: strerror_r(-524)=22
  libbpf: prog 'test_ptr_struct_arg': failed to auto-attach: -524
  test_map_val_and_probed_memory:FAIL:test_ldsx_insn__attach unexpected error: -524 (errno 524)
  #116/1   ldsx_insn/map_val and probed_memory:FAIL
  #116/2   ldsx_insn/ctx_member_sign_ext:OK
  #116/3   ldsx_insn/ctx_member_narrow_sign_ext:OK
  #116     ldsx_insn:FAIL

  All error logs:
  test_map_val_and_probed_memory:PASS:test_ldsx_insn__open 0 nsec
  test_map_val_and_probed_memory:PASS:test_ldsx_insn__load 0 nsec
  libbpf: prog 'test_ptr_struct_arg': failed to attach: ERROR: strerror_r(-524)=22
  libbpf: prog 'test_ptr_struct_arg': failed to auto-attach: -524
  test_map_val_and_probed_memory:FAIL:test_ldsx_insn__attach unexpected error: -524 (errno 524)
  #116/1   ldsx_insn/map_val and probed_memory:FAIL
  #116     ldsx_insn:FAIL
  Summary: 0/2 PASSED, 0 SKIPPED, 1 FAILED

Signed-off-by: Hengqi Chen <[email protected]>
Signed-off-by: Huacai Chen <[email protected]>
matttbe pushed a commit that referenced this issue May 28, 2024
Add a test case to assert that the skb->pkt_type which was set from the BPF
program is retained from the netkit xmit side to the peer's device at tcx
ingress location.

  # ./vmtest.sh -- ./test_progs -t netkit
  [...]
  ./test_progs -t netkit
  [    1.140780] bpf_testmod: loading out-of-tree module taints kernel.
  [    1.141127] bpf_testmod: module verification failed: signature and/or required key missing - tainting kernel
  [    1.284601] tsc: Refined TSC clocksource calibration: 3408.006 MHz
  [    1.286672] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x311fd9b189d, max_idle_ns: 440795225691 ns
  [    1.290384] clocksource: Switched to clocksource tsc
  #345     tc_netkit_basic:OK
  #346     tc_netkit_device:OK
  #347     tc_netkit_multi_links:OK
  #348     tc_netkit_multi_opts:OK
  #349     tc_netkit_neigh_links:OK
  #350     tc_netkit_pkt_type:OK
  Summary: 6/0 PASSED, 0 SKIPPED, 0 FAILED

Signed-off-by: Daniel Borkmann <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Alexei Starovoitov <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement sched packets scheduler
Projects
Status: Needs triage
Development

No branches or pull requests

4 participants