Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use an object for option keys #8080

Merged
merged 17 commits into from
Jan 4, 2021

Conversation

dcbaker
Copy link
Member

@dcbaker dcbaker commented Dec 7, 2020

This series introduces a new OptionKey type, which is a bit like a NamedTuple, but with some nice helper methods. The class itself is immutable (or, at least as immutable as a tuple), pickleable, sortable, and comparable. It stores 5 public traits: the name, the target machine, the subproject, the language (if it's a compiler option), and the type (project, base, builtin, compiler, etc). The thing that makes it super nice to work with the the evolve() method, which takes an existing OptionKey, and creates a new copy with specific fields overwritten in the copy, ala:

a = OptionKey('default_library', subproject='')
b = a.evolve(subproject='sub1')

It has a few convenience methods for common cases like:

a.as_root() == a.evolve(subproject='')
a.as_build() == a.evolve(machine=MachineChoice.BUILD)

It also provides a from_string() alternative constructor, which is parses command line string format, and creates an OptionKey:

a = OptionKey.from_string('foo:build.c_std')  # I might have the foo and build backwards here
assert a.name == std
assert a.subproject == foo
assert a.machine is MachineChoice.BUILD
assert a.lang == 'c'

Equally importantly, it provides a way back to the string form by converting it to a string:

a = OptionKey('std', subproject='foo', machine=MachineChoice.BUILD, lang='c')
str(a) == 'foo:build.c_std'

It also stores it's hash privately (which is safe to do because it's immutable), which makes it very fast to find in dictionaries or hashes.

There is a lot of code churn here, but there's also a lot of really ugly code that goes away as a result, no need for the code that walks over all of the different option dictionaries to figure out where an option is. Nor do we need all of the crazy code to reconstruct the string form from the internal form, and it centralizes the string -> class conversion code in a single place. We also get away from "stringly" typing.

I'm expecting a few failures in CI still (particularly on windows as I haven't run the VS backend yet).

@lgtm-com
Copy link

lgtm-com bot commented Dec 7, 2020

This pull request introduces 7 alerts when merging 6efed37 into e828c67 - view on LGTM.com

new alerts:

  • 4 for Unused import
  • 1 for Unused local variable
  • 1 for Incomplete ordering
  • 1 for Module is imported with 'import' and 'import from'

@dcbaker dcbaker force-pushed the submit/option-key-type branch from 31744e0 to 3bd598e Compare December 7, 2020 22:53
@lgtm-com
Copy link

lgtm-com bot commented Dec 7, 2020

This pull request introduces 1 alert when merging dee56b4 into e828c67 - view on LGTM.com

new alerts:

  • 1 for Unused import

@dcbaker dcbaker force-pushed the submit/option-key-type branch 9 times, most recently from 81ebdb1 to e3129ef Compare December 14, 2020 18:44
@dcbaker dcbaker force-pushed the submit/option-key-type branch from e3129ef to e58fdad Compare December 23, 2020 21:49
@dcbaker dcbaker force-pushed the submit/option-key-type branch from 4bf3928 to f3df202 Compare January 4, 2021 19:22
@jpakkane
Copy link
Member

jpakkane commented Jan 4, 2021

Traceback (most recent call last):
  File "D:\a\1\s\mesonbuild\mesonmain.py", line 132, in run
    return options.run_func(options)
  File "D:\a\1\s\mesonbuild\msetup.py", line 275, in run
    app.generate()
  File "D:\a\1\s\mesonbuild\msetup.py", line 183, in generate
    self._generate(env)
  File "D:\a\1\s\mesonbuild\msetup.py", line 245, in _generate
    intr.backend.generate()
  File "D:\a\1\s\mesonbuild\backend\ninjabackend.py", line 534, in generate
    self.generate_target(t)
  File "D:\a\1\s\mesonbuild\backend\ninjabackend.py", line 864, in generate_target
    o, s = self.generate_single_compile(target, src, False, [], header_deps)
  File "D:\a\1\s\mesonbuild\backend\ninjabackend.py", line 2559, in generate_single_compile
    self.add_dependency_scanner_entries_to_element(target, compiler, element)
  File "D:\a\1\s\mesonbuild\backend\ninjabackend.py", line 2566, in add_dependency_scanner_entries_to_element
    if not self.should_use_dyndeps_for_target(target):
  File "D:\a\1\s\mesonbuild\backend\ninjabackend.py", line 897, in should_use_dyndeps_for_target
    if self.environment.coredata.compiler_options[target.for_machine]['cpp']['std'] != 'c++latest':
AttributeError: 'CoreData' object has no attribute 'compiler_options'

This seems to be in the new C++ module scanner code that is only exercised under Visual Studio 2019 preview.

This is pretty important to be able to debug the test, as it's huge and
really should be a test split into subtests.
Otherwise bugs like "option c_args is unknown" can slip through. that's
bad.
This is a complex key that can store multiple bits of data in a single
place. It can be generated from a command line formatted string, and
it's str method returns it to that form.

It's intentionally immutable, use the evolve() method to create
variations of an existing key.
that's the only place it's used anyway.
This is useful for figuring out what kind of option this is. My hope is
that in the longterm this is less useful, but we'll still want it for
the configuration summary printing.
they're probably not strictly needed, but it makes mypy happy.
@dcbaker dcbaker force-pushed the submit/option-key-type branch from f3df202 to d5880f5 Compare January 4, 2021 20:18
There's starting to be a lot of things including coredata that coredata
needs to itself include. putting it in mesonlib makes more sense
I would have prefered to do these seperatately, but they are combined in
some cases, so it was much easier to convert them together.

this eliminates the builtins_per_machine dict, as it's duplicated with
the OptionKey's machine parameter.
This patches takes the options work to it's logical conclusion: A single
flat dictionary of OptionKey: UserOptions. This allows us to simplify a
large number of cases, as we don't need to check if an option is in this
dict or that one (or any of 5 or 6, actually).
The output of list_targets is a pretty horrific jumble of things. We
really need a TypeDict to make this not so terrible we can't deal with
it, so for now just use Any.
@dcbaker dcbaker force-pushed the submit/option-key-type branch from d5880f5 to f14bf8b Compare January 4, 2021 20:21
@dcbaker
Copy link
Member Author

dcbaker commented Jan 4, 2021

@jpakkane: fixed. Just cygwin failing because pip isn't getting installed.

@jpakkane jpakkane merged commit d47a5c8 into mesonbuild:master Jan 4, 2021
@dcbaker dcbaker deleted the submit/option-key-type branch January 5, 2021 01:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants