Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor option getting v2 #13441

Open
wants to merge 104 commits into
base: master
Choose a base branch
from
Open

Refactor option getting v2 #13441

wants to merge 104 commits into from

Conversation

jpakkane
Copy link
Member

@jpakkane jpakkane commented Jul 17, 2024

This is basically a rebased version of #13086. Well, half of it currently. Will do the rest after this.

The refactor operation turned out to be so big I though it better to open a new one and leave that MR exist in its original state in case it is ever needed for debugging. Will close that once this is merged.

This has a bunch of broken commits and fixing them would be too much work, so this must be merged with a merge commit rather than rebasing.

@jpakkane jpakkane requested a review from dcbaker as a code owner July 17, 2024 16:37
@jpakkane jpakkane marked this pull request as draft July 17, 2024 16:37
mesonbuild/coredata.py Fixed Show fixed Hide fixed
unittests/optiontests.py Dismissed Show dismissed Hide dismissed
mesonbuild/options.py Fixed Show fixed Hide fixed
mesonbuild/coredata.py Fixed Show fixed Hide fixed
mesonbuild/mintro.py Fixed Show fixed Hide fixed
mesonbuild/backend/ninjabackend.py Fixed Show fixed Hide fixed
mesonbuild/build.py Fixed Show fixed Hide fixed
mesonbuild/compilers/c.py Fixed Show fixed Hide fixed
mesonbuild/options.py Fixed Show fixed Hide fixed
mesonbuild/options.py Fixed Show fixed Hide fixed
@jpakkane
Copy link
Member Author

Cherry-picked the relevant remaining commits. Expect everything to be still broken. Will work on fixing things again next.

optstore = OptionStore()
name = 'cpp_std'
sub_name = 'sub'
sub2_name = 'sub2'

Check notice

Code scanning / CodeQL

Unused local variable Note

Variable sub2_name is not used.
mesonbuild/interpreter/kwargs.py Fixed Show fixed Hide fixed
mesonbuild/mconf.py Fixed Show fixed Hide fixed
mesonbuild/dependencies/qt.py Fixed Show fixed Hide fixed
unittests/optiontests.py Fixed Show fixed Hide fixed
unittests/optiontests.py Fixed Show fixed Hide fixed
unittests/optiontests.py Fixed Show fixed Hide fixed
unittests/optiontests.py Fixed Show fixed Hide fixed
mesonbuild/options.py Fixed Show fixed Hide fixed
mesonbuild/options.py Fixed Show fixed Hide fixed
mesonbuild/options.py Fixed Show fixed Hide fixed
mesonbuild/options.py Fixed Show fixed Hide fixed
unittests/optiontests.py Fixed Show fixed Hide fixed
mesonbuild/coredata.py Fixed Show fixed Hide fixed
@jpakkane
Copy link
Member Author

jpakkane commented Jul 28, 2024

Fixed enough that all common tests pass. Feel free to start reviewing those bits.

No point in commenting on typing, though, it has not been done at all.

mesonbuild/options.py Fixed Show fixed Hide fixed
mesonbuild/options.py Fixed Show fixed Hide fixed
@jpakkane
Copy link
Member Author

Now passes all project tests.

On my machine.

@jpakkane jpakkane marked this pull request as ready for review July 31, 2024 15:06
@jpakkane jpakkane requested a review from mensinda as a code owner July 31, 2024 15:06
unittests/optiontests.py Fixed Show fixed Hide fixed
unittests/optiontests.py Fixed Show fixed Hide fixed
unittests/optiontests.py Fixed Show fixed Hide fixed
unittests/optiontests.py Fixed Show fixed Hide fixed
unittests/optiontests.py Fixed Show fixed Hide fixed
unittests/optiontests.py Fixed Show fixed Hide fixed
unittests/optiontests.py Fixed Show fixed Hide fixed
unittests/optiontests.py Fixed Show fixed Hide fixed
unittests/optiontests.py Fixed Show fixed Hide fixed
unittests/optiontests.py Fixed Show fixed Hide fixed
unittests/optiontests.py Fixed Show fixed Hide fixed
mesonbuild/coredata.py Fixed Show fixed Hide fixed
mesonbuild/mesonmain.py Fixed Show fixed Hide fixed
mesonbuild/mesonmain.py Fixed Show fixed Hide fixed
def hard_reset_from_prefix(self, prefix:str):
for optkey, prefix_mapping in BUILTIN_DIR_NOPREFIX_OPTIONS.items():
valobj = self.options[optkey]
new_value = valobj.value

Check warning

Code scanning / CodeQL

Variable defined multiple times Warning

This assignment to 'new_value' is unnecessary as it is
redefined
before this value is used.
This assignment to 'new_value' is unnecessary as it is
redefined
before this value is used.
@jpakkane jpakkane force-pushed the optionrefactor2 branch 3 times, most recently from 609bf4a to 8111038 Compare August 19, 2024 18:54
mesonbuild/compilers/cpp.py Fixed Show fixed Hide fixed
mesonbuild/compilers/cpp.py Fixed Show fixed Hide fixed
mesonbuild/compilers/cpp.py Fixed Show fixed Hide fixed
general version of enabling optimizations on all subprojects but not
the top project would be done like this:

meson configure -Doptimization=2 -A:optimization=0
Copy link
Member

@dnicolodi dnicolodi Aug 21, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should be meson configure -Doptimization=0 -A:optimization=2 with the values passed to the optimization option swapped (this applies to your recent blog post too).

Copy link
Member Author

@jpakkane jpakkane Aug 21, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not really:

-Doptimization=2 -A:optimization=0 means "optimize everything except the top level project.

-Doptimization=0 -A:optimization=2 means "optimize top project only".

The former is you want most of the time and what happens when you use Linux distro dependencies. The -A:bob syntax means apply only to the top level project, not apply to all subprojects.

If this syntax is confusing we can change it. I did go through a bunch of them and this seemed to be the most consistant and simple, but if someone comes up with a better syntax, great.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Uhm. I indeed misunderstood the syntax for the -A option argument. After thinking for a brief moment, I think the syntax implemented here makes sense.

However, I'm not sure of the reason why both the -A and -D options are needed. I think that meson configure -Doptimization=0 -A:optimization=2 could be written as meson configure -Doptimization=0 -D:optimization=2 and that the argument to the second -D option could be parsed to make it into something that applies to the top project only (valid option names cannot contain columns, AFAIK). Having two syntax for specifying setup options feels strange. Similarly, the -U option would apply both to global options ans to project specific options.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The idea was that -A is used to create a new augment, -U is for unsetting it and -D is for changing the value of an existing thing. So for example:

meson configure -Asub:opt=v

This creates a new augment and then doing

meson configure -Asub:opt=v

is an error, because creating an augment that already exists would be a logical error. Then doing

meson configure -Dsub:opt=w

would change the value,

meson configure -Usub:opt

would delete it and after that doing

meson configure -Dsub:opt=w

would be an error again, as the subproject option does not exist.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why does the action of creating an augment need to be explicit, from the point of view of the user? I'm also not sure about how it combines with meson setup --reconfigure. Should -A or -D be used to set an augment when running meson setup --reconfigure? This is particularly confusing because recent meson versions accent the --reconfigure option also when the specified build directory is not yet initialized.

Currently there is no way to reset an option to the project default, thus it would be nice to be able to use the -U option to unset auments and global options.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think probably the common mental model that most people have of options as they are now, before this PR, is:

  • -Dfoo:bar=true sets the value of an existing option in a subproject, -Dbar=true sets the value of an existing option in the superproject
  • options specified in a meson_options.txt / meson.options file only exist for a single project (maybe the superproject)
  • options built into meson create an option for every subproject, but some of them are read-only aliases of the superproject version of the option, such as prefix, and some of them are distinct settable options such as default_library
  • using get_option('bar') in a meson.build file will look up the subproject option, even if it is a read-only alias.

The part of this mental model we are trying to fix is the fact that:

  • "what is a read-only alias and what is actually settable" is hugely inconsistent and we want most things to be settable
  • there's no straightforward way to set a per-subproject option without enumerating all subprojects plus the superproject

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't taken a good look at how this PR handles it all, but one straightforward way to define the distinction is by looking at something like prefix.

With this PR, what happens if you try to "augment" the prefix option?

With this PR, what option gets looked up when you do get_option('prefix') in a subproject? What is the user-facing mental model of that option? This scenario doesn't appear to be covered by the unittests, at least, and it should.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a commit that removes -A in favour of -D. Docs not updates. Does this make the UI better?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ping @dnicolodi.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for being receptive of my comments @jpakkane. I haven't had the occasion to delve into the implementation, and my previous comments were based almost entirely on reading the additions to the documentation, which is not yet updated here. However, having only the -D option clearly solves most of my issue with the new feature.

a single subproject called `numbercruncher` that does heavy
computation. During development you want to build that subproject with
optimizations enabled but your main project without
optimizations. This can be done by specifying an augment to the given
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The first tow times I read "augment" I thought it was "argument" misspelled. IIUC, augments are subproject-specific options. Can't the concept be called "subproject-specific option" instead of "augment"? I think it would be clearer and I don't think it need to be written so often for the lengthy name to be a problem.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Originally I wanted to call it "override", but that is already used for per-target option values, so no. I guess we can call them "subproject specific options" in docs, but internally we need a more concise name. And in any case it would be nice to have the same name for this feature internally and externally.

"Augment" was the best I could come up with, better ideas welcome.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dumb thought: call it an augmentation instead of an augment and no one will get it confused with an argument?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"Augment" was the best I could come up with, better ideas welcome.

Supersede or Negate might be considered. But expanding them to "supersedes" or "Negates" or "Negations" fouls things up. Then there is supersession, where the main advantage is that it's a long way from `argument'. ;)

@jpakkane jpakkane force-pushed the optionrefactor2 branch 2 times, most recently from 72a484a to 70f2c67 Compare November 6, 2024 12:30
mesonbuild/arglist.py Fixed Show fixed Hide fixed
@jpakkane jpakkane force-pushed the optionrefactor2 branch 2 times, most recently from fd617cb to 1a9f046 Compare November 6, 2024 13:04
docs/markdown/snippets/optionrefactor.md Outdated Show resolved Hide resolved
docs/markdown/snippets/optionrefactor.md Outdated Show resolved Hide resolved
docs/markdown/snippets/optionrefactor.md Outdated Show resolved Hide resolved
@jpakkane
Copy link
Member Author

jpakkane commented Nov 9, 2024

Fixed docs.

@jpakkane
Copy link
Member Author

One more things to note is that we can add a subproject_default_options kwarg to project so that projects can specify only the things needed when built as a subproject (such as setting c_std to gnu99 or smth).-

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants