BASED ON #517 [#525] resizable shm support #527

elfenpiff · 2024-11-25T19:29:46Z

Notes for Reviewer

Makes the ZeroCopyConnection concept multi segment capable. Instead of using of UsedChunkList, that tracks all outgoing samples to recover them when a subscriber dies, we have now one UsedChunkList per segment.
The user can define how many segments at most the ZeroCopyConnection can manage in parallel. This is useful to avoid the overhead in the case that one has a static type and not the slice case as payload.

Pre-Review Checklist for the PR Author

Checklist for the PR Reviewer

Commits are properly organized and messages are according to the guideline
Unit tests have been written for new behavior
Public API is documented
PR title describes the changes

Post-review Checklist for the PR Author

All open points are addressed and tracked via issues

References

Closes #525

…on layer

…new requirements

…e suggested by SlotMap

…fail when the maximum amount of reallocations was exceeded

…ist problem

…used segments

…opy connection

…instead of usize

…ple segment ids

codecov · 2024-11-25T23:22:39Z

Codecov Report

Attention: Patch coverage is 87.55365% with 87 lines in your changes missing coverage. Please review.

Project coverage is 79.11%. Comparing base (2bfd0c9) to head (f656be3).
Report is 25 commits behind head on main.

Files with missing lines	Patch %	Lines
...ceoryx2-cal/src/resizable_shared_memory/dynamic.rs	88.67%	48 Missing ⚠️
iceoryx2-cal/src/zero_copy_connection/common.rs	84.61%	14 Missing ⚠️
iceoryx2-cal/src/shm_allocator/mod.rs	78.37%	8 Missing ⚠️
iceoryx2-bb/container/src/slotmap.rs	62.50%	6 Missing ⚠️
iceoryx2-bb/container/src/semantic_string.rs	50.00%	3 Missing ⚠️
iceoryx2-bb/lock-free/src/spsc/index_queue.rs	70.00%	3 Missing ⚠️
...ck-free/src/spsc/safely_overflowing_index_queue.rs	84.61%	2 Missing ⚠️
iceoryx2-cal/src/shm_allocator/pool_allocator.rs	96.00%	2 Missing ⚠️
iceoryx2-cal/src/shm_allocator/bump_allocator.rs	97.05%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #527      +/-   ##
==========================================
- Coverage   79.14%   79.11%   -0.04%     
==========================================
  Files         201      202       +1     
  Lines       24142    24826     +684     
==========================================
+ Hits        19108    19640     +532     
- Misses       5034     5186     +152

Files with missing lines	Coverage Δ
...l/src/communication_channel/posix_shared_memory.rs	`90.64% <100.00%> (ø)`
...yx2-cal/src/communication_channel/process_local.rs	`89.31% <100.00%> (ø)`
iceoryx2-cal/src/dynamic_storage/mod.rs	`100.00% <ø> (ø)`
...yx2-cal/src/dynamic_storage/posix_shared_memory.rs	`89.07% <100.00%> (-0.22%)`	⬇️
iceoryx2-cal/src/shared_memory/common.rs	`86.53% <100.00%> (+0.15%)`	⬆️
iceoryx2-cal/src/shared_memory/mod.rs	`100.00% <ø> (ø)`
iceoryx2-cal/src/shared_memory_directory/file.rs	`86.36% <100.00%> (ø)`
iceoryx2-cal/src/zero_copy_connection/mod.rs	`12.50% <ø> (ø)`
iceoryx2/src/port/details/publisher_connections.rs	`98.68% <100.00%> (ø)`
...ceoryx2/src/port/details/subscriber_connections.rs	`94.80% <100.00%> (ø)`
... and 12 more

... and 4 files with indirect coverage changes

… path

elfenpiff added 30 commits November 19, 2024 16:30

[eclipse-iceoryx#497] Introduce iceoryx2-fal, the framework abstracti…

fa4954a

…on layer

[eclipse-iceoryx#497] Introduce resizable shared memory

610c319

[eclipse-iceoryx#497] Implementing resizable shared memory

3eef147

[eclipse-iceoryx#497] Integrate SlotMap

1f84877

[eclipse-iceoryx#497] Add generic Allocator argument

561a756

[eclipse-iceoryx#497] Remove iceoryx2_fal; create first tests

7b32343

[eclipse-iceoryx#497] Allocator can provide hint to adjust config to …

cfe2535

…new requirements

[eclipse-iceoryx#497] New segments are created when out-of-memory

c3dcb1c

[eclipse-iceoryx#497] View remaps new segments

4846ded

[eclipse-iceoryx#497] Add chunk counting

7b7f255

[eclipse-iceoryx#497] Introduce SlotMap::next_free_key

af43b6f

[eclipse-iceoryx#497] SegmentId does not need to increment and will b…

d29a8bd

…e suggested by SlotMap

[eclipse-iceoryx#497] Provide max chunk layout hint

2ad50ee

[eclipse-iceoryx#497] Increase SegmentId size to 16-bit

59011fd

[eclipse-iceoryx#497] SegmentId is incrementing and allocations will …

a535a38

…fail when the maximum amount of reallocations was exceeded

[eclipse-iceoryx#497] Fix bitshift issue in PointerOffset

b9f11ce

[eclipse-iceoryx#497] Test increasing chunk size allocation

856121d

[eclipse-iceoryx#497] Verify allocation strategy

9416a69

[eclipse-iceoryx#497] Add allocator tests

99be5f8

[eclipse-iceoryx#497] Finalize pool allocator hint tests

d63a821

[eclipse-iceoryx#497] Finalize bump allocator hint tests

98af279

[eclipse-iceoryx#497] Implement list, does_exist and remove

45d743c

[eclipse-iceoryx#497] Test does_exist, list and remove

679d6a1

[eclipse-iceoryx#504] Open complex segments

c317e68

[eclipse-iceoryx#497] Create separate shared state

b75d49e

[eclipse-iceoryx#497] Introduce dummy mgmt segment to solve shm dir l…

f475e07

…ist problem

[eclipse-iceoryx#497] Add alignment test

f266f68

[eclipse-iceoryx#497] Separate builder for view and memory

620426b

[eclipse-iceoryx#497] Document ResizableSharedMemory

d92f5c1

[eclipse-iceoryx#497] Fix clippy warnings

e62f622

elfenpiff added 7 commits November 19, 2024 16:30

[eclipse-iceoryx#497] Update changelog

c1c9bd1

[eclipse-iceoryx#497] Add tests that verify that the view releases un…

1d6d6c5

…used segments

[eclipse-iceoryx#497] Add documentation to ShmAllocator

f574878

[eclipse-iceoryx#525] Add basic untested segment id support to zero c…

f54b496

…opy connection

[eclipse-iceoryx#525] index queue is based on u64 instead of usize

017c879

[eclipse-iceoryx#525] safely overflowing index queue is based on u64 …

b538316

…instead of usize

[eclipse-iceoryx#525] Test multi segment usage in zero copy connection

5f51073

elfenpiff requested review from orecham and elBoberido November 25, 2024 19:29

elfenpiff self-assigned this Nov 25, 2024

elfenpiff changed the title ~~BASED ON [#525] resizable shm support~~ BASED ON 517 [#525] resizable shm support Nov 25, 2024

elfenpiff changed the title ~~BASED ON 517 [#525] resizable shm support~~ BASED ON #517 [#525] resizable shm support Nov 25, 2024

[eclipse-iceoryx#525] Finalize tests for zero copy channel with multi…

5dfcfae

…ple segment ids

elfenpiff force-pushed the iox2-525-resizable-shm-support branch from 50d399e to 5dfcfae Compare November 25, 2024 19:31

[eclipse-iceoryx#525] Remove unnecessary set_log_level calls

5cb60e2

elfenpiff added 2 commits November 26, 2024 00:31

[eclipse-iceoryx#525] Add i386 ubuntu dependencies

1bf0bfd

[eclipse-iceoryx#525] Fix 32-bit int size bug

f656be3

elfenpiff force-pushed the iox2-525-resizable-shm-support branch from 53a6c76 to f656be3 Compare November 26, 2024 01:04

[eclipse-iceoryx#525] verify_init() does not use format! macro on hot…

c9decdb

… path

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

BASED ON #517 [#525] resizable shm support #527

BASED ON #517 [#525] resizable shm support #527

elfenpiff commented Nov 25, 2024 •

edited

Loading

codecov bot commented Nov 25, 2024 •

edited

Loading

BASED ON #517 [#525] resizable shm support #527

Are you sure you want to change the base?

BASED ON #517 [#525] resizable shm support #527

Conversation

elfenpiff commented Nov 25, 2024 • edited Loading

Notes for Reviewer

Pre-Review Checklist for the PR Author

Checklist for the PR Reviewer

Post-review Checklist for the PR Author

References

codecov bot commented Nov 25, 2024 • edited Loading

Codecov Report

elfenpiff commented Nov 25, 2024 •

edited

Loading

codecov bot commented Nov 25, 2024 •

edited

Loading