aws: fork mbedtls 2.25.0 base64 utility to avoid 2.26.0+ performance hit

[matthewfala]

Migrate to performant 2.25.0-mbedtls base64 implementation.

Tested via unit tests + 1 end to end test
Sent logs to firehose via the out_kinesis_firehose plugin which utilizes the implementation.

---

Enter [N/A] in the box, if an item is not applicable to your change.

Testing
Before we can approve your change; please submit the following in a comment:

• [N/A] Example configuration file for the change
• [N/A] Debug log output from testing the change

• [N/A] Attached Valgrind output that shows no leaks or memory corruption was found (No memory allocations)

Documentation

• Documentation required for this feature

---

Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.

[PettitWesley]

@edsiper I'd like your review/approval on this as well.

[edsiper]

@PettitWesley base64 unit test is failing

https://github.com/fluent/fluent-bit/runs/4478384757?check_suite_focus=true#step:5:1810

[edsiper]

Why not to make it generic use instead of just aws scoped? I think it could be useful as flb_base64.c alone

[matthewfala]

Strange how it passes on some operating systems. Grateful for the crossplatform unit test runner. I'll take a look. I can refactor this to flb_base64

Since I did some refactoring to the aws unit test structure for the base64 test, I might switch to an aws placeholder test that noops, so future tests to aws can be added easily if that is alright.

[matthewfala]

@PettitWesley Could you take a look at the changes? 3c08a68

I think I resolved the problem with the testcase, it may have just been a lack of a null character when calling strlen. I initialize the array to 0 now in the decode test case.

[PettitWesley]

@matthewfala CI says the tests are still failing? I am good to approve/merge once everything passes.

[matthewfala]

Need to refactor the commit messages, then this should be done

[matthewfala]

@edsiper Could you please take a look at this when you get the chance?

[nokute78]

@matthewfala @PettitWesley @krispraws
Before revewing

• Could you fix conflict ?
• How much did that improve ?
• How about upgrading mbedtls v2.28.0 since the patch Backport 2.2x: range-based constant-flow base64 Mbed-TLS/mbedtls#4835 has been merged ? Mbed-TLS/mbedtls@8e8251a

[edsiper]

thanks @nokute78 for the hints about mbedtls, I will make sure to upgrade in the 1.8 branch

[matthewfala]

Hi @nokute78,
Actually, could you let me know if this pr will be considered? Then I'll fix the conflicts.

According to Ramya's testing as described in the following issue:
#4110
The performance hits of Mbedtls's new base64 function is severe, causing significant drop in throughput in our plugin that uses mbedtls.

It appears the newest base64 implementation is still slower than the older implementation as described by the changelog message in ChangeLog.d/base64-ranges.txt

Changes
   * Improve the performance of base64 constant-flow code. The result is still
     slower than the original non-constant-flow implementation, but much faster
     than the previous constant-flow implementation. Fixes #4814.

Would forking be considered above just upgrading mbedtls?

[nokute78]

@matthewfala In my opinion, forking can be a risk and take a porting cost. e.g. Watching upstream and merging patches.
If the performance using mbedtls v2.28 is not acceptable for user/developer, we should merge this PR as an interim patch.
If it is not, we need to upgrade mbedtls.

How much slower is it using mbedtls v2.28 ?

[matthewfala]

Thank you @nokute78. I like the interim patch idea. I'll fix the merge conflicts. We can assess the speed difference after, though I am not familiar with profiling, so I'll either learn or ask someone else on the team do help.

[matthewfala]

@nokute78 Could you take a look? Resolved merge conflicts

[nokute78]

Could you add test case to compare performance between flb_base64_* and mbedtls_base64_* ?
The test fails when mbedtls_base64_* are faster than flb_base64_* or the performance of mbedtls_base64_* becomes acceptable.

We can detect it is the time to drop the interim patch and back to mbedtls.

[nokute78]

@matthewfala Thank you for updating.
LGTM.

[matthewfala]

@nokute78 YW. Could you give me an eta on when this will be merged?

[nokute78]

@matthewfala I can't merge since "Merging can be performed automatically with 1 approving review." .
@edsiper Could you merge this ?

[edsiper]

@matthewfala would you please submit a PR for 1.8 branch ?

[matthewfala]

Definitely! Will publish a pr to 1.8 today.

[matthewfala]

Here is the backport PR!: #4801