extccomp clean up and refactorings

[alaviss]

extccomp right now is a mess. It's functions are:

• Provide details about the target C compiler: how to execute them, what features it supports, etc.
• Compile C/C++/ASM files.
• Produce shared libraries, static libraries and executable depending on compiler flags.
• Manage nimcache for the C backend.

On top of that, these functions are mingled with one another with no clear boundaries and the code quality is plainly embarrassing.

I think we should start splitting this module into several parts, and here is my rough idea for it:

• Global cache system (compilercache): Basically nimcache, but formalized into a proper API

  • Keep track of assets as well as what could cause them to be invalidated

• C compiler information (ccompinfo):

  • How to invoke them
    • Basic rules to generate objects, etc.
  • What C features/extensions do they support
    • Ideally should be dynamically computed. Right now the information is hard-coded. This leads to issues like -d:nimEmulateOverflowChecks has to be used for older gcc.
    • Need a caching system to support dynamically computed data as it is used heavily by codegen.
    • If done properly we might be able to remove nimbase.h. Not sure if desirable.
    • (Maybe) Provide an user API into this system.

• Build rules generation (buildrules): This could be expanded to support external builders (ie. for csources generation), but I don't think that's high priority.

  • What objects has to be generated
    • Build dependencies
  • How to generate these objects
    • (Maybe) Basic generation rule (ie. .c -> .o)
      • Allow objects to override the rule (ie. adding more flags)

• Builder (builder):

  • Verify that files are up-to-date (from data provided by build rules)
  • Execute the rules

Known "features" in today's extccomp that we might be able to reuse:

• nim jsonbuild: Compile a nim program from json script produced by the compiler. This builder is conceived for dealing with the compiler using too much RAM.
• .json cache manifest/build rule: This special file serves both as the cache manifest as well as the build rule. The format is pretty inflexible and doesn't leave much data for outside tooling, but it was never meant to be for that
• --genMapping: A hidden feature designed for niminst to create csources. It has limited support for .nim with {.passC.} or {.passL.} directives nor has any support for additional include paths from --cincludes.

These are just ideas right now. We have to discuss the specifics on how to get this done and whether it is a priority.