Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sync master #2

Merged
merged 562 commits into from
Jun 14, 2021
Merged

sync master #2

merged 562 commits into from
Jun 14, 2021

Conversation

llhhbc
Copy link
Owner

@llhhbc llhhbc commented Jun 14, 2021


Enter [N/A] in the box, if an item is not applicable to your change.

Testing
Before we can approve your change; please submit the following in a comment:

  • Example configuration file for the change
  • Debug log output from testing the change
  • Attached Valgrind output that shows no leaks or memory corruption was found

Documentation

  • Documentation required for this feature

Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.

nokute78 and others added 30 commits March 5, 2021 18:23
- replace whitelist with allowlist
  - keep Whitelist_key option for backwards compatibility
  - add allowlist_key option as alias of Whitelist_key

Signed-off-by: Takahiro Yamashita <[email protected]>
Do not assume the existence of OpenSSL on Windows machines,
but rather includes it in the binary using static linking.

This fixes the issue "Fluent Bit fails to start up due to
missing libcrypto/libssl DLLs".

Signed-off-by: Fujimoto Seiji <[email protected]>
* Added a new unit-tests.yaml workflow
    * Updated workflow documentation
    * Removed travis-ci from the repository
    * Removed the ci/* directory and moved into fluent-bit-ci

Signed-off-by: Jorge Niedbalski <[email protected]>
Signed-off-by: Eduardo Silva <[email protected]>
Summary of Changes

1. A new cmake file, ./cmake/linux-arm64.cmake, to configure the build
to use the aarch64-linux-gnu toolchain.
2. Changes in ./CMakeLists.txt to update the CONFIGURE and BUILD
commands for libbacktrace and jemalloc.
3. Changes in ./cmake/luajit.cmake and ./cmake/onigmo.cmake to update
the CONFIGURE and BUILD commands for luajit and onigmo libraries.
Fluentbit for arm64 using the cross-compilation enabled by changes

Benefits
1. The build time using native cross-compilation for arm64 is much
faster than that using QEMU userspace emulation, which is the current
method used by OSS to release multi-arch images. In our experiments, it
was ~20X faster (~1min vs. ~20min).
2. Supporting cross-platform builds natively will allow users to set up
builds for multiple architectures in their build pipelines without
supplying additional configurations or Dockerfiles.
3. We will be able to easily support additional architectures (e.g.
armv7) cross-compilation by setting the right toolchain path during
cmake.

Signed-off-by: Ziyuan Chen <[email protected]>
Add a linter to check commit message.

Signed-off-by: Jorge Niedbalski <[email protected]>
Enable integration tests to run on GCP:
integration tests will run on gcp on a
1.19, 1.20 cluster.

Signed-off-by: Jorge Niedbalski <[email protected]>
Signed-off-by: Eduardo Silva <[email protected]>
Use microk8s integration only for development
of new integration tests.

Signed-off-by: Jorge Niedbalski <[email protected]>
Switches checkallcommitmessages to false 
to only check last commit.
We need to pass the --host to libbacktrace in order for it to
cross-compile correctly. Also, we need some additional environment
variables in cmake/linux-arm64.cmake to ensure the dependencies are
linked correctly during a Docker build.

Tested this locally and on a Kubernetes cluster.

Signed-off-by: Ziyuan Chen <[email protected]>
Updated the build-release file to use docker buildx. This way, we
can natively cross-compile multi-arch docker images. Updated
dockerfiles/Dockerfile.arm64v8 to use native cross-compilation instead
of QEMU emulation.

Docker buildx is compatible with docker build commands, so existing
builds should not be affected.

Signed-off-by: Ziyuan Chen <[email protected]>
master branch should run from x86_64-master tag, not from latest PR.
This commit adds the workflow status badges into the README.md page.
This patch makes http_user/password configuration
the priority, if unset, it will default to splunk_token.

If splunk_token && http_user is unset, it will return
an error.

Signed-off-by: Jorge Niedbalski <[email protected]>
If integration succeeds, adds a grafana tag with the commit id/message.
prevent adding commit tag before running integration tests.
add a task for clone ci repository on publish_images
This is a follow-up PR to
fluent/fluent-bit-ci#6

Signed-off-by: Jorge Niedbalski <[email protected]>
Signed-off-by: Jorge Niedbalski <[email protected]>
This commit enables the benchmark
suite on master branch.

Signed-off-by: Jorge Niedbalski <[email protected]>
force apt update on unit tests 18.04
niedbalski and others added 27 commits May 25, 2021 22:48
Signed-off-by: Jorge Niedbalski <[email protected]>
…olve the query locally and return without creating a socket

There was a corner case where c-ares would resolve a name locally (in the reported case it was a ipv4 address) and dispatch the result callback immediately without creating a socket or sending a query (obvious). In that case, the lookup routine would hang after yielding waiting for the event loop handler to resume it after receiving the result.
This patch selectively calls yield only when a socket was created after the c-ares api call is issued.

This fix could fail if we tried to force c-ares to use TCP sockets and keep them alive and reuse them but in that case the whole thing fails so I don't think that is something to worry about (well, don't do that of course).

Signed-off-by: Leonardo Alminana <[email protected]>
Signed-off-by: Jorge Niedbalski <[email protected]>
    If set, the value of the key will be handled as array.
    It is useful to handle empty array.

Signed-off-by: Takahiro Yamashita <[email protected]>
Signed-off-by: Eduardo Silva <[email protected]>
On LuaJIT, it requests to specify MACOSX_DEPLOYMENT_TARGET explicitly on
build command arguments.

Signed-off-by: Hiroshi Hatake <[email protected]>
This is because, since Mojave (10.14), macOS SDK does not put headers in
/usr/include.
Thus, we need to tell where is sysroot that includes headers and
libraries.

Signed-off-by: Hiroshi Hatake <[email protected]>
@llhhbc llhhbc changed the title sync master Jun 14, 2021
@llhhbc llhhbc merged commit 2a32cba into llhhbc:master Jun 14, 2021
llhhbc pushed a commit that referenced this pull request Sep 15, 2021
output_thread patch #1 : pipe creation is platform dependent, in unixes it's
achieved through the pipe syscall but in windows it's a socket pair created
through libevent which means instead of read/write we need to use recv/send
which was already abstracted through flb_pipe_(r/w).

output_thread patch #2 : in windows the SOCKET data type is defined as an
UINT_PTR which means in 64 bit operating systems it's an 8 byte number
instead of 4. libevent abstracts this through the evutil_socket_t data type
which in turn is abstracted using the flb_pipefd_t data type in fluent bit.

The problem comes to play when calling mk_event_channel_create which
receives 2 int pointers to return both pipe endpoints. This means there
are 2 possible bugs (one of which was happening and the other is not
really a concern I think) :

Since mk_event_channel_create was only modifying the low part of the elements
in the ch_parent_events array and the structure was not zeroed when allocated,
the high parts of those 64 bit members contained garbage which caused any
winsock calls to return error 10038 (not a socket). That's the reason why I
added a memset call in line 421.

There is still a possible issue with mk_event_channel_create which we should
fix by moving our platform dependent data type to mk_lib and defining our
flb local data types to those if need be (for consistency).

Signed-off-by: Leonardo Alminana <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.