Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add ebpf() plugin and reuseport randomization #4365

Merged
merged 8 commits into from
Mar 20, 2023

Conversation

bazsi
Copy link
Collaborator

@bazsi bazsi commented Mar 9, 2023

ebpf() plugin and udp packet balancer: a new ebpf() plugin was added which
leverages the kernel's eBPF infrastructure to improve performance and
scalability of syslog-ng. The first ebpf based solution improves
performance when a single (or very few) senders generate most of the inbound
UDP traffic that syslog-ng needs to process. Normally, the kernel
distributes load between so-reuseport sockets by keeping each flow (e.g.
same source/dest ip/port) in its dedicated receiver. This fails to balance
the sockets properly if only a few senders are responsible for most of the
load. ebpf(reuseport()) will replace the original kernel algorithm with an
alternative that changes the algorithm, so individual packets will be
assigned to one of the sockets randomly, thereby producing a more uniform
load.

Example:

source s_udp {
        udp(so-reuseport(yes) port(2000) persist-name("udp1")
                ebpf(reuseport(sockets(4)))
        );
        udp(so-reuseport(yes) port(2000) persist-name("udp2"));
        udp(so-reuseport(yes) port(2000) persist-name("udp3"));
        udp(so-reuseport(yes) port(2000) persist-name("udp4"));
};

Depends on #4373

@bazsi bazsi marked this pull request as draft March 9, 2023 18:16
@bazsi bazsi changed the title Randomize receive socket for incoming UDP streams WIP: Randomize receive socket for incoming UDP streams Mar 9, 2023
@kira-syslogng
Copy link
Contributor

Build FAILURE

@bazsi bazsi force-pushed the ebpf-randomize-udp-streams branch from 7a27366 to 3316f54 Compare March 10, 2023 16:08
@kira-syslogng
Copy link
Contributor

Build FAILURE

@bazsi bazsi force-pushed the ebpf-randomize-udp-streams branch from 3316f54 to 68d046f Compare March 11, 2023 07:24
@kira-syslogng
Copy link
Contributor

Build FAILURE

@bazsi bazsi force-pushed the ebpf-randomize-udp-streams branch from 68d046f to 136bff2 Compare March 11, 2023 13:40
@kira-syslogng
Copy link
Contributor

Build FAILURE

@bazsi bazsi force-pushed the ebpf-randomize-udp-streams branch 5 times, most recently from ec2adc8 to acf0bdf Compare March 15, 2023 08:50
@bazsi bazsi changed the title WIP: Randomize receive socket for incoming UDP streams add ebpf() plugin and reuseport randomization Mar 15, 2023
@bazsi bazsi marked this pull request as ready for review March 15, 2023 08:52
@bazsi
Copy link
Collaborator Author

bazsi commented Mar 15, 2023

This PR depends on some packages that we need to install into our tarball container, but the CI is run on a previous version. At the moment, I've added the installation of these packages in our actions workflow in a temporary commit. As soon as we are ready with the review, we should probably merge the dbld pieces first, wait for images to be generated and then merge the rest, while dropping the temporary change in the actions script.

Copy link
Collaborator

@alltilla alltilla left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have found some build/configure related things.

Some of them were not trivial for me to understand and recommend a fix, so I have fixed them on my working branch and opened a PR for your convenience: bazsi#17

Could you kindly check it out? Thanks!

news/feature-4365.md Outdated Show resolved Hide resolved
configure.ac Outdated Show resolved Hide resolved
modules/ebpf/CMakeLists.txt Outdated Show resolved Hide resolved
modules/ebpf/CMakeLists.txt Outdated Show resolved Hide resolved
@alltilla
Copy link
Collaborator

The dbld image dep PR went in, we can now rebase this to master and drop the workaround commit.

This hook can be used by plugins to potentially change socket related
settings once syslog-ng has finished by its own.

This makes it possible to attach EBPF programs, change socket options, etc.

Signed-off-by: Balazs Scheidler <[email protected]>
@bazsi bazsi force-pushed the ebpf-randomize-udp-streams branch from acf0bdf to 0b46933 Compare March 18, 2023 06:38
modules/ebpf/CMakeLists.txt Outdated Show resolved Hide resolved
@alltilla
Copy link
Collaborator

Only one comment, otherwise LGTM.

@czanik
Copy link
Collaborator

czanik commented Mar 20, 2023

Could you add some developer notes to the NEWS about the configure options and necessary dependencies?

Signed-off-by: Balazs Scheidler <[email protected]>
Building the eBPF module is not for the fainthearted, but we need some
testing, so enable it in the devshell build, even if it is disabled
everywhere else.  This ensures that at least we are in a compilable state.

Signed-off-by: Balazs Scheidler <[email protected]>
@bazsi bazsi force-pushed the ebpf-randomize-udp-streams branch from ea5051f to 875d108 Compare March 20, 2023 13:00
Signed-off-by: Balazs Scheidler <[email protected]>
Signed-off-by: Attila Szakacs <[email protected]>
Signed-off-by: Balazs Scheidler <[email protected]>
@bazsi bazsi force-pushed the ebpf-randomize-udp-streams branch from 875d108 to ffde0e4 Compare March 20, 2023 13:12
@bazsi
Copy link
Collaborator Author

bazsi commented Mar 20, 2023

I've added a paragraph to the NEWS file. in short don't compile it, unless you need it. Also, it will be a nightmare to enable this in all distros.

@alltilla alltilla merged commit 61d8189 into syslog-ng:master Mar 20, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants