tls: Load system certificates on Windows

[jschwartzentruber]

Signed-off-by: Jesse Schwartzentruber [email protected]

This change ports the code from the mbed implementation to iterate over the Windows CA store and load each into the OpenSSL CA store.

Fixes #4735

---

Enter [N/A] in the box, if an item is not applicable to your change.

Testing
Before we can approve your change; please submit the following in a comment:

• [N/A] Example configuration file for the change
• Debug log output from testing the change

In the below case I ran in another terminal: echo hello world > live.log
I confirmed this line is reported to Google Cloud Logs properly.

[2022/02/04 08:02:35] [ info] [engine] started (pid=4244)
[2022/02/04 08:02:35] [debug] [engine] coroutine stack size: 98302 bytes (96.0K)
[2022/02/04 08:02:35] [debug] [storage] [cio stream] new stream registered: tail.0
[2022/02/04 08:02:35] [ info] [storage] version=1.1.6, initializing...
[2022/02/04 08:02:35] [ info] [storage] in-memory
[2022/02/04 08:02:35] [ info] [storage] normal synchronization mode, checksum disabled, max_chunks_up=128
[2022/02/04 08:02:35] [ info] [cmetrics] version=0.2.2
[2022/02/04 08:02:36] [debug] [input:tail:tail.0] flb_tail_fs_stat_init() initializing stat tail input
[2022/02/04 08:02:36] [debug] [input:tail:tail.0] 0 new files found on path 'C:\Users\User\Desktop\live.log'
[2022/02/04 08:02:36] [debug] [input:tail:tail.0] 0 new files found on path 'C:\Users\User\Desktop\screenlog.*'
[2022/02/04 08:02:36] [debug] [storage] [cio stream] new stream registered: emitter.1
[2022/02/04 08:02:36] [debug] [stackdriver:stackdriver.0] created event channels: read=716 write=720
[2022/02/04 08:02:36] [ info] [output:stackdriver:stackdriver.0] metadata_server set to http://metadata.google.internal
[2022/02/04 08:02:37] [debug] [output:stackdriver:stackdriver.0] JWT signature: xxx
[2022/02/04 08:02:37] [debug] [http_client] not using http_proxy for header
[2022/02/04 08:02:37] [ info] [oauth2] HTTP Status=200
[2022/02/04 08:02:37] [debug] [oauth2] payload: xxx
[2022/02/04 08:02:37] [ info] [oauth2] access token from 'www.googleapis.com:443' retrieved
[2022/02/04 08:02:37] [debug] [upstream] KA connection #796 to www.googleapis.com:443 is now available
[2022/02/04 08:02:37] [debug] [file:file.1] created event channels: read=848 write=852
[2022/02/04 08:02:37] [debug] [router] match rule tail.0:stackdriver.0
[2022/02/04 08:02:37] [debug] [router] match rule emitter.1:stackdriver.0
[2022/02/04 08:02:37] [ info] [sp] stream processor started
[2022/02/04 08:02:37] [debug] [input:tail:tail.0] [static files] processed 0b, done
[2022/02/04 08:02:42] [debug] [input:tail:tail.0] 0 new files found on path 'C:\Users\User\Desktop\live.log'
[2022/02/04 08:02:42] [debug] [input:tail:tail.0] 0 new files found on path 'C:\Users\User\Desktop\screenlog.*'
[2022/02/04 08:02:47] [debug] [input:tail:tail.0] 0 new files found on path 'C:\Users\User\Desktop\live.log'
[2022/02/04 08:02:47] [debug] [input:tail:tail.0] 0 new files found on path 'C:\Users\User\Desktop\screenlog.*'
[2022/02/04 08:02:52] [debug] [input:tail:tail.0] 0 new files found on path 'C:\Users\User\Desktop\live.log'
[2022/02/04 08:02:52] [debug] [input:tail:tail.0] 0 new files found on path 'C:\Users\User\Desktop\screenlog.*'
[2022/02/04 08:02:57] [debug] [input:tail:tail.0] 0 new files found on path 'C:\Users\User\Desktop\live.log'
[2022/02/04 08:02:57] [debug] [input:tail:tail.0] 0 new files found on path 'C:\Users\User\Desktop\screenlog.*'
[2022/02/04 08:03:02] [debug] [input:tail:tail.0] 0 new files found on path 'C:\Users\User\Desktop\live.log'
[2022/02/04 08:03:02] [debug] [input:tail:tail.0] 0 new files found on path 'C:\Users\User\Desktop\screenlog.*'
[2022/02/04 08:03:07] [debug] [input:tail:tail.0] inode=1970324837213267 with offset=0 appended as C:\Users\User\Desktop\live.log
[2022/02/04 08:03:07] [debug] [input:tail:tail.0] 1 new files found on path 'C:\Users\User\Desktop\live.log'
[2022/02/04 08:03:07] [debug] [input:tail:tail.0] 0 new files found on path 'C:\Users\User\Desktop\screenlog.*'
[2022/02/04 08:03:07] [debug] [upstream] drop keepalive connection #-1 to www.googleapis.com:443 (keepalive idle timeout)
[2022/02/04 08:03:07] [debug] [input chunk] update output instances with new chunk size diff=65
[2022/02/04 08:03:07] [debug] [input:tail:tail.0] [static files] processed 16b
[2022/02/04 08:03:07] [debug] [input:tail:tail.0] inode=1970324837213267 file=C:\Users\User\Desktop\live.log promote to TAIL_EVENT
[2022/02/04 08:03:07] [debug] [input:tail:tail.0] [static files] processed 0b, done
[2022/02/04 08:03:12] [debug] [task] created task=0000021F5F943DC0 id=0 OK
[2022/02/04 08:03:12] [debug] [input:tail:tail.0] 0 new files found on path 'C:\Users\User\Desktop\live.log'
[2022/02/04 08:03:12] [debug] [input:tail:tail.0] 0 new files found on path 'C:\Users\User\Desktop\screenlog.*'
[2022/02/04 08:03:13] [debug] [output:stackdriver:stackdriver.0] [logging.googleapis.com/monitored_resource] not found in the payload
[2022/02/04 08:03:13] [debug] [http_client] not using http_proxy for header
[2022/02/04 08:03:13] [debug] [output:stackdriver:stackdriver.0] HTTP Status=200
[2022/02/04 08:03:13] [debug] [upstream] KA connection #948 to logging.googleapis.com:443 is now available
[2022/02/04 08:03:13] [debug] [out flush] cb_destroy coro_id=0
[2022/02/04 08:03:13] [debug] [task] destroy task=0000021F5F943DC0 (task_id=0)
[2022/02/04 08:03:17] [debug] [input:tail:tail.0] 0 new files found on path 'C:\Users\User\Desktop\live.log'

With windows_load_system_certificates commented out, same machine and config.
In this case I again ran echo hello world 2 >> live.log and confirmed this line was not reported to Google Cloud Logs.

[2022/02/04 08:07:19] [ info] [engine] started (pid=5672)
[2022/02/04 08:07:19] [debug] [engine] coroutine stack size: 98302 bytes (96.0K)
[2022/02/04 08:07:19] [debug] [storage] [cio stream] new stream registered: tail.0
[2022/02/04 08:07:19] [ info] [storage] version=1.1.6, initializing...
[2022/02/04 08:07:19] [ info] [storage] in-memory
[2022/02/04 08:07:19] [ info] [storage] normal synchronization mode, checksum disabled, max_chunks_up=128
[2022/02/04 08:07:19] [ info] [cmetrics] version=0.2.2
[2022/02/04 08:07:19] [debug] [input:tail:tail.0] flb_tail_fs_stat_init() initializing stat tail input
[2022/02/04 08:07:19] [debug] [input:tail:tail.0] inode=1970324837213267 with offset=16 appended as C:\Users\User\Desktop\live.log
[2022/02/04 08:07:19] [debug] [input:tail:tail.0] 1 new files found on path 'C:\Users\User\Desktop\live.log'
[2022/02/04 08:07:19] [debug] [input:tail:tail.0] 0 new files found on path 'C:\Users\User\Desktop\screenlog.*'
[2022/02/04 08:07:19] [debug] [storage] [cio stream] new stream registered: emitter.1
[2022/02/04 08:07:19] [debug] [stackdriver:stackdriver.0] created event channels: read=732 write=736
[2022/02/04 08:07:19] [ info] [output:stackdriver:stackdriver.0] metadata_server set to http://metadata.google.internal
[2022/02/04 08:07:19] [debug] [output:stackdriver:stackdriver.0] JWT signature: xxx
[2022/02/04 08:07:20] [debug] [upstream] connection #784 failed to www.googleapis.com:443
[2022/02/04 08:07:20] [debug] [upstream] connection #800 failed to www.googleapis.com:443
[2022/02/04 08:07:20] [error] [oauth2] could not get an upstream connection to www.googleapis.com:443
[2022/02/04 08:07:20] [error] [output:stackdriver:stackdriver.0] error retrieving oauth2 access token
[2022/02/04 08:07:20] [ warn] [output:stackdriver:stackdriver.0] token retrieval failed
[2022/02/04 08:07:20] [debug] [file:file.1] created event channels: read=804 write=808
[2022/02/04 08:07:20] [debug] [router] match rule tail.0:stackdriver.0
[2022/02/04 08:07:20] [debug] [router] match rule emitter.1:stackdriver.0
[2022/02/04 08:07:20] [ info] [sp] stream processor started
[2022/02/04 08:07:20] [debug] [input:tail:tail.0] inode=1970324837213267 file=C:\Users\User\Desktop\live.log promote to TAIL_EVENT
[2022/02/04 08:07:20] [debug] [input:tail:tail.0] [static files] processed 0b, done
[2022/02/04 08:07:25] [debug] [input:tail:tail.0] 0 new files found on path 'C:\Users\User\Desktop\live.log'
[2022/02/04 08:07:25] [debug] [input:tail:tail.0] 0 new files found on path 'C:\Users\User\Desktop\screenlog.*'
[2022/02/04 08:07:28] [debug] [input chunk] update output instances with new chunk size diff=67
[2022/02/04 08:07:30] [debug] [input:tail:tail.0] 0 new files found on path 'C:\Users\User\Desktop\live.log'
[2022/02/04 08:07:30] [debug] [input:tail:tail.0] 0 new files found on path 'C:\Users\User\Desktop\screenlog.*'
[2022/02/04 08:07:30] [debug] [task] created task=000001C91C397620 id=0 OK
[2022/02/04 08:07:31] [debug] [upstream] connection #800 failed to logging.googleapis.com:443
[2022/02/04 08:07:31] [debug] [out flush] cb_destroy coro_id=0
[2022/02/04 08:07:31] [debug] [retry] new retry created for task_id=0 attempts=1
[2022/02/04 08:07:31] [ warn] [engine] failed to flush chunk '5672-1643990848.343034000.flb', retry in 6 seconds: task_id=0, input=emitter_for_rewrite_tag.0 > output=stackdriver.0 (out_id=0)
[2022/02/04 08:07:35] [debug] [input:tail:tail.0] 0 new files found on path 'C:\Users\User\Desktop\live.log'
[2022/02/04 08:07:35] [debug] [input:tail:tail.0] 0 new files found on path 'C:\Users\User\Desktop\screenlog.*'
[2022/02/04 08:07:37] [debug] [upstream] connection #884 failed to logging.googleapis.com:443
[2022/02/04 08:07:37] [debug] [out flush] cb_destroy coro_id=1
[2022/02/04 08:07:37] [debug] [task] task_id=0 reached retry-attempts limit 1/1
[2022/02/04 08:07:37] [ warn] [engine] chunk '5672-1643990848.343034000.flb' cannot be retried: task_id=0, input=emitter_for_rewrite_tag.0 > output=stackdriver.0
[2022/02/04 08:07:37] [debug] [task] destroy task=000001C91C397620 (task_id=0)
[2022/02/04 08:07:40] [debug] [input:tail:tail.0] 0 new files found on path 'C:\Users\User\Desktop\live.log'
[2022/02/04 08:07:40] [debug] [input:tail:tail.0] 0 new files found on path 'C:\Users\User\Desktop\screenlog.*'

• [N/A] Attached Valgrind output that shows no leaks or memory corruption was found

If this is a change to packaging of containers or native binaries then please confirm it works for all targets.

• [N/A] Attached local packaging test output showing all targets (including any new ones) build.

Documentation

• [N/A] Documentation required for this feature

Backporting

• Backport to latest stable release.

1.8: #4752

---

Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.

[leonardo-albertovich]

Once these issues are addressed it will have my blessing. Obviously the same thing goes for the 1.8 port which is the exact same code.

[leonardo-albertovich]

I've approved this PR and the backport one as well.