Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

http_server: v2: reload: windows: Enable hot reloading #7984

Merged
merged 1 commit into from
Oct 5, 2023

Conversation

cosmo0920
Copy link
Contributor

@cosmo0920 cosmo0920 commented Sep 28, 2023

Building on top of calyptia-fleet works on Windows, we can also offer for enabling hot-reloading on Windows. :)

However, Windows does not have SIGHUP signal. So, I enabled only for HTTP API for reloading.

Note that GenerateConsoleCtrlEvent returns nonzero when succeeded.

ref: https://learn.microsoft.com/en-us/windows/console/generateconsolectrlevent


Enter [N/A] in the box, if an item is not applicable to your change.

Testing
Before we can approve your change; please submit the following in a comment:

  • Example configuration file for the change
PS> bin/fluent-bit -i random -o stdout -H -P 2020 -Y -v
  • Debug log output from testing the change

On Another Terminal

PS>  curl -XPOST -d '{}' http://localhost:2020/api/v2/reload
{"reload":"done","status":1}

Fluent Bit Terminal

Fluent Bit v2.1.10
* Copyright (C) 2015-2022 The Fluent Bit Authors
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
* https://fluentbit.io

[2023/09/28 12:40:59] [ info] Configuration:
[2023/09/28 12:40:59] [ info]  flush time     | 1.000000 seconds
[2023/09/28 12:40:59] [ info]  grace          | 5 seconds
[2023/09/28 12:40:59] [ info]  daemon         | 0
[2023/09/28 12:40:59] [ info] ___________
[2023/09/28 12:40:59] [ info]  inputs:
[2023/09/28 12:40:59] [ info]      random
[2023/09/28 12:40:59] [ info] ___________
[2023/09/28 12:40:59] [ info]  filters:
[2023/09/28 12:40:59] [ info] ___________
[2023/09/28 12:40:59] [ info]  outputs:
[2023/09/28 12:40:59] [ info]      stdout.0
[2023/09/28 12:40:59] [ info] ___________
[2023/09/28 12:40:59] [ info]  collectors:
[2023/09/28 12:40:59] [ info] [fluent bit] version=2.1.10, commit=b19e9ce674, pid=27336
[2023/09/28 12:40:59] [debug] [engine] coroutine stack size: 98302 bytes (96.0K)
[2023/09/28 12:40:59] [ info] [storage] ver=1.5.1, type=memory, sync=normal, checksum=off, max_chunks_up=128
[2023/09/28 12:40:59] [ info] [cmetrics] version=0.6.3
[2023/09/28 12:40:59] [ info] [ctraces ] version=0.3.1
[2023/09/28 12:40:59] [ info] [input:random:random.0] initializing
[2023/09/28 12:40:59] [ info] [input:random:random.0] storage_strategy='memory' (memory only)
[2023/09/28 12:40:59] [debug] [random:random.0] created event channels: read=848 write=852
[2023/09/28 12:40:59] [debug] [input:random:random.0] interval_sec=1 interval_nsec=0
[2023/09/28 12:40:59] [debug] [stdout:stdout.0] created event channels: read=856 write=860
[2023/09/28 12:40:59] [ info] [http_server] listen iface=0.0.0.0 tcp_port=2020
[2023/09/28 12:40:59] [ info] [sp] stream processor started
[2023/09/28 12:40:59] [ info] [output:stdout:stdout.0] worker #0 started
[2023/09/28 12:41:00] [debug] [input chunk] update output instances with new chunk size diff=50, records=1, input=random.0
[2023/09/28 12:41:01] [debug] [task] created task=000001FB96EA2920 id=0 OK
[0] random.0: [[1695872460.781617500, {}], {"rand_value"=>[2023/09/28 12:41:01] [debug] [output:stdout:stdout.0] task_id=0 assigned to thread #0
3917686252860891653}]
[2023/09/28 12:41:01] [debug] [input chunk] update output instances with new chunk size diff=50, records=1, input=random.0
[2023/09/28 12:41:01] [debug] [out flush] cb_destroy coro_id=0
[2023/09/28 12:41:01] [debug] [task] destroy task=000001FB96EA2920 (task_id=0)
[2023/09/28 12:41:02] [debug] [task] created task=000001FB96EA2920 id=0 OK
[2023/09/28 12:41:02] [debug] [output:stdout:stdout.0] task_id=0 assigned to thread #0
[0] random.0: [[1695872461.776284700, [2023/09/28 12:41:02] [debug] [input chunk] update output instances with new chunk size diff=50, records=1, input=random.0
{}], {"rand_value"=>8350676561454476613}]
[2023/09/28 12:41:02] [debug] [out flush] cb_destroy coro_id=1
[2023/09/28 12:41:02] [debug] [task] destroy task=000001FB96EA2920 (task_id=0)
[2023/09/28 12:41:03] [debug] [task] created task=000001FB96EA2920 id=0 OK
[0] random.0: [[1695872462.786276300, {}], {"rand_value"=>5297848698850845187}]
[2023/09/28 12:41:03] [debug] [output:stdout:stdout.0] task_id=0 assigned to thread #0
[2023/09/28 12:41:03] [debug] [input chunk] update output instances with new chunk size diff=50, records=1, input=random.0
[2023/09/28 12:41:03] [debug] [out flush] cb_destroy coro_id=2
[2023/09/28 12:41:03] [debug] [task] destroy task=000001FB96EA2920 (task_id=0)
[2023/09/28 12:41:04] [debug] [task] created task=000001FB96EA2920 id=0 OK
[0] random.0: [[1695872463.786365700, {}], {"rand_value"=>13289125307504265363}]
[2023/09/28 12:41:04] [debug] [output:stdout:stdout.0] task_id=0 assigned to thread #0
[2023/09/28 12:41:04] [debug] [input chunk] update output instances with new chunk size diff=50, records=1, input=random.0
[2023/09/28 12:41:04] [debug] [out flush] cb_destroy coro_id=3
[2023/09/28 12:41:04] [debug] [task] destroy task=000001FB96EA2920 (task_id=0)
[2023/09/28 12:41:05] [debug] [task] created task=000001FB96EA2920 id=0 OK
[0] random.0: [[1695872464.781742700, {}], {"rand_value"=>7913022013376780552}]
[2023/09/28 12:41:05] [debug] [output:stdout:stdout.0] task_id=0 assigned to thread #0
[2023/09/28 12:41:05] [debug] [input chunk] update output instances with new chunk size diff=50, records=1, input=random.0
[2023/09/28 12:41:05] [debug] [out flush] cb_destroy coro_id=4
[2023/09/28 12:41:05] [debug] [task] destroy task=000001FB96EA2920 (task_id=0)
[2023/09/28 12:41:06] [debug] [task] created task=000001FB96EA2920 id=0 OK
[0] random.0: [[1695872465.781798000, {}], {"rand_value"=>5159549540827597325}]
[2023/09/28 12:41:06] [debug] [output:stdout:stdout.0] task_id=0 assigned to thread #0
[2023/09/28 12:41:06] [debug] [out flush] cb_destroy coro_id=5
[2023/09/28 12:41:06] [debug] [input chunk] update output instances with new chunk size diff=50, records=1, input=random.0
[2023/09/28 12:41:06] [debug] [task] destroy task=000001FB96EA2920 (task_id=0)
[2023/09/28 12:41:06] [ info] reloading instance pid=27336 tid=000001FB96EC2A90
[2023/09/28 12:41:06] [ info] [reload] stop everything of the old context
[2023/09/28 12:41:06] [debug] [output:stdout:stdout.0] task_id=0 assigned to thread #0
[2023/09/28 12:41:06] [ warn] [engine] service will shutdown when all remaining tasks are flushed
[0] random.0: [[1695872466.781374300, {}], {"rand_value"=>18407437581212968048}]
[2023/09/28 12:41:06] [ info] [input] pausing random.0
[2023/09/28 12:41:07] [ info] [engine] service has stopped (0 pending tasks)
[2023/09/28 12:41:07] [ info] [input] pausing random.0
[2023/09/28 12:41:07] [ info] [output:stdout:stdout.0] thread worker #0 stopping...
[2023/09/28 12:41:07] [ info] [output:stdout:stdout.0] thread worker #0 stopped
[2023/09/28 12:41:09] [ info] [reload] start everything
[2023/09/28 12:41:09] [ info] [fluent bit] version=2.1.10, commit=b19e9ce674, pid=27336
[2023/09/28 12:41:09] [debug] [engine] coroutine stack size: 98302 bytes (96.0K)
[2023/09/28 12:41:09] [ info] [storage] ver=1.5.1, type=memory, sync=normal, checksum=off, max_chunks_up=128
[2023/09/28 12:41:09] [ info] [cmetrics] version=0.6.3
[2023/09/28 12:41:09] [ info] [ctraces ] version=0.3.1
[2023/09/28 12:41:09] [ info] [input:random:random.0] initializing
[2023/09/28 12:41:09] [ info] [input:random:random.0] storage_strategy='memory' (memory only)
[2023/09/28 12:41:09] [debug] [random:random.0] created event channels: read=1140 write=1144
[2023/09/28 12:41:09] [debug] [input:random:random.0] interval_sec=1 interval_nsec=0
[2023/09/28 12:41:09] [debug] [stdout:stdout.0] created event channels: read=1148 write=1152
[2023/09/28 12:41:09] [ info] [http_server] listen iface=0.0.0.0 tcp_port=2020
[2023/09/28 12:41:09] [ info] [sp] stream processor started
[2023/09/28 12:41:09] [ info] [output:stdout:stdout.0] worker #0 started
[2023/09/28 12:41:10] [debug] [input chunk] update output instances with new chunk size diff=50, records=1, input=random.0
[2023/09/28 12:41:11] [debug] [task] created task=000001FB96F5C9E0 id=0 OK
[0] random.0: [[1695872470.879987800, {}], {"rand_value"=>3196691273628167248}]
[2023/09/28 12:41:11] [debug] [output:stdout:stdout.0] task_id=0 assigned to thread #0
[2023/09/28 12:41:11] [debug] [input chunk] update output instances with new chunk size diff=50, records=1, input=random.0
[2023/09/28 12:41:11] [debug] [out flush] cb_destroy coro_id=0
[2023/09/28 12:41:11] [debug] [task] destroy task=000001FB96F5C9E0 (task_id=0)
[2023/09/28 12:41:12] [debug] [task] created task=000001FB96F5CA80 id=0 OK
[2023/09/28 12:41:12] [debug] [output:stdout:stdout.0] task_id=0 assigned to thread #0
[0] random.0: [[1695872471.888636300, {}], {"rand_value"=>12186872785323929565}]
  • Attached Valgrind output that shows no leaks or memory corruption was found

If this is a change to packaging of containers or native binaries then please confirm it works for all targets.

  • Run local packaging test showing all targets (including any new ones) build.
  • Set ok-package-test label to test for all targets (requires maintainer to do).

Documentation

  • Documentation required for this feature

fluent/fluent-bit-docs#1209

Backporting

  • Backport to latest stable release.

Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.

Building on top of calyptia-fleet works on Windows,
we can also offer for enabling hot-reloading on Windows. :)

Signed-off-by: Hiroshi Hatake <[email protected]>
@cosmo0920 cosmo0920 temporarily deployed to pr September 28, 2023 03:44 — with GitHub Actions Inactive
@cosmo0920 cosmo0920 temporarily deployed to pr September 28, 2023 03:44 — with GitHub Actions Inactive
@cosmo0920 cosmo0920 temporarily deployed to pr September 28, 2023 03:44 — with GitHub Actions Inactive
@cosmo0920 cosmo0920 temporarily deployed to pr September 28, 2023 04:11 — with GitHub Actions Inactive
@edsiper edsiper merged commit da569ea into master Oct 5, 2023
45 of 47 checks passed
@edsiper edsiper deleted the cosmo0920-enable-hot-reloading-on-windows branch October 5, 2023 20:04
leonardo-albertovich pushed a commit that referenced this pull request Nov 3, 2023
Building on top of calyptia-fleet works on Windows,
we can also offer for enabling hot-reloading on Windows. :)

Signed-off-by: Hiroshi Hatake <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants