Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make kernel not to use boost program_options library #1014

Closed
wkozaczuk opened this issue Nov 12, 2018 · 4 comments
Closed

Make kernel not to use boost program_options library #1014

wkozaczuk opened this issue Nov 12, 2018 · 4 comments

Comments

@wkozaczuk
Copy link
Collaborator

wkozaczuk commented Nov 12, 2018

OSv kernel uses following parts of boost libraries:

  • C++ collection and utilities templates mostly used in core/.cc; given corresponding instantiations for each type parameter combination become part of relevant object files, it is not clear how much it contributes to the size of code sections (text) of loader.elf (it seems it should be possible to measure it somehow though bloaty is not capable to show it)
    • boost/algorithm/cxx11/all_of.hpp
    • boost/algorithm/string.hpp
    • boost/algorithm/string/replace.hpp
    • boost/algorithm/string/split.hpp
    • boost/config/warning_disable.hpp
    • boost/date_time.hpp
    • boost/dynamic_bitset.hpp
    • boost/format.hpp
    • boost/intrusive/list.hpp
    • boost/intrusive/parent_from_member.hpp
    • boost/intrusive/set.hpp
    • boost/lockfree/policies.hpp
    • boost/lockfree/queue.hpp
    • boost/lockfree/stack.hpp
    • boost/range/adaptor/reversed.hpp
    • boost/range/algorithm/find.hpp
    • boost/range/algorithm/remove.hpp
    • boost/range/algorithm/transform.hpp
    • boost/spirit/include/qi.hpp
    • boost/utility.hpp
    • boost/variant.hpp
  • libboost_system.a - tiny (10K code and 40K total) library providing error handling
  • libboost_program_options.a - fairly large (400K code and 1.5M total) library to handle command line options and parameters

Unlike boost C++ templates that are used in many places of code and provide critical functionality that would be hard to replace with something hand-written, program options is only used by loader.cc and core/commands.cc and given its substantial size it is desirable to either replace it with something smaller or somehow make use of this library optional. Based on some experiments involving removing programs options completely from kernels it was shown that resulting loader-stripped.elf becomes smaller by 500-600K.

The solution I am proposing is hybrid:

  • rewrite loader.cc::parse_options() by using getopt_long() (which is already part of musl in OSv) instead of boost::program_options
  • extract commands.cc::parse_command_line(const std::string line, bool& ok) and related code as an optional library commands.so that would be used dynamically if found on filesystem; new commands.so would be linked dynamically against libboost_program_options.so; move options parsing code (osv::parse_cmdline()) from commands.cc to new options.cc
  • change loader.cc:prepare_commands() to dynamically use commands.so if present to parse command line or fall back to a simple implementation that handle cmdline as single application

Please note that parse_command_line() is called by loader.cc after VFS is fully setup so nothing prevents us from loading commands.so from filesystem.

@nyh
Copy link
Contributor

nyh commented Nov 29, 2018

I think dropping boost program_options should also be done for cpiod (see issue #980), not just in the main kernel.
Note that are additional parts we use from Boost which is not listed above. One example I particularly disliked (but don't know how much overhead it really adds) is Boost Qi, used in commands.cc.
I think we should only consider moving command.cc to a separate shared object if it turns out that this Boost Qi thing is very large and we cannot improve on it.

@wkozaczuk
Copy link
Collaborator Author

I did look at this again and measured the size impact of boost spirit QI and it seems to increase the size of loader-stripped.elfby 56K only. So program options is the big winner.
Also removing program options from kernel may help us fix the "hidden" issue - #1040 - less to hide :-)

@wkozaczuk
Copy link
Collaborator Author

wkozaczuk commented Sep 28, 2019

Please note that using getopt_long in loader.cc might be overkill as well given all options handled are in form --option=value so parsing this manually is actually not that hard.

wkozaczuk added a commit to wkozaczuk/osv that referenced this issue Oct 3, 2019
…tions

This patch changes loader.cc to use light options utility
instead of boost program options.

This patch partially addresses cloudius-systems#1014 and together with one
of the previous patches sent to update core/commands.cc
to remove boost program options, fully addresses cloudius-systems#1014.

Signed-off-by: Waldemar Kozaczuk <[email protected]>
nyh pushed a commit that referenced this issue Oct 3, 2019
…tions

This patch changes loader.cc to use light options utility
instead of boost program options.

This patch partially addresses #1014 and together with one
of the previous patches sent to update core/commands.cc
to remove boost program options, fully addresses #1014.

Signed-off-by: Waldemar Kozaczuk <[email protected]>
Message-Id: <[email protected]>
nyh pushed a commit that referenced this issue Oct 3, 2019
This patch partially addresses #1014 and removes usage
of boost program_options library in core/commands.cc.
More specifically it reimplements handling of runscript
options: '--env=VAR=abc' by using simple logic to detect and parse
individual argv elements.

In order to fully address #1014 we also need to modify loader.cc
to not use program options.

Signed-off-by: Waldemar Kozaczuk <[email protected]>
Message-Id: <[email protected]>
@wkozaczuk
Copy link
Collaborator Author

All the 3 commits referencing this issue have addressed this issue so I am closing it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants