Evaluate using Link-Time Optimization (LTO) and Profile-Guided Optimization (PGO) for the project

[zamazan4ik]

Hi!

I want to discuss several compiler optimizations that could be helpful for the project. Maybe you even already tested them and have some benchmark numbers to share publicly ;)

Did you test Link-Time Optimization (LTO) for Sqlean modules? If yes, do you have any numbers regarding performance and the resulting binary size? It would be nice to test since enabling LTO is not too time-consuming thing (usually, huh).

If we are talking about more advanced things, recently I started evaluating using Profile-Guided Optimization (PGO) for various applications and workloads (including databases like PostgreSQL, ClickHouse, SQLite, MySQL and many others) - all the results are available at https://github.com/zamazan4ik/awesome-pgo . For PGO and SQLite you check this forum thread. It would be interesting to test PGO not only for SQLite itself but for such modules too to gain some additional performance wins.

Post-Link Optimization (PLO) is similar to PGO technique but uses a slightly different approach. You can take a look at LLVM BOLT for more information. However, I recommend starting playing with PLO only after applying PGO - PGO is a much more stable and time-proven technology compared to PLO tools.

By the way, I see that for several files -O1 is used instead of -O2/-O3. Could you please elaborate a bit - why such a decision was made? I guess it's connected somehow to the constant-time calculations or other "fragile" (I mean "vulnerable to dangerous compiler optimizations") but it's just a guess. Maybe LTO wasn't enabled for the same reason...

Thank you.

P.S. Since Discussions are disabled for the repository, I created an issue instead. Please don't treat it like a bug or smth like that.

[nalgeon]

Hi Alexander! Thank you for your interest in Sqlean.

I haven't tried LTO, PGO or PLO.

O2/O3 causes test failures for some extensions, so I use O1 for them.