Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

use a template for computing output_path for packageable targets #21175

Merged
merged 12 commits into from
Jul 17, 2024

Conversation

tdyas
Copy link
Contributor

@tdyas tdyas commented Jul 15, 2024

As described in #18659, when a user sets output_path on a packageable target, the user gives up the default behavior which uses a normalized version of the target's directory as the directory into which the place the built artifact. While the user could replicate the behavior manually, it would be a better user experience if the user can still have partial access to some of the default behavior.

This PR teaches the output_path field how to derive its value from a template instead of hard-coding its behavior in Python. The output_path field learns the following template replacements:

  • spec_path_normalized: The path to the target's directory ("spec path") with forward slashes replaced by dots.
  • target_name_normalized: The target's name
  • file_suffix: For target's which produce single file artifacts, this is the file type suffix to use. This is empty if not applicable.

The default value for the output_path field is now "${spec_path_normalized}/${target_name_normalized}${file_suffix}" which replicates the existing behavior.

string.Template is used as the template engine.

This PR also fixes some incorrect usages of output_path in several backends.

@tdyas tdyas force-pushed the interpolate_artifact_name branch 2 times, most recently from 8696810 to e9ca2ec Compare July 15, 2024 20:48
@tdyas tdyas force-pushed the interpolate_artifact_name branch from e9ca2ec to 80dd7b3 Compare July 15, 2024 20:49
@@ -106,7 +106,8 @@ async def pack_node_package_into_tgz_for_publication(
),
)
if field_set.output_path.value:
digest = await Get(Digest, AddPrefix(result.output_digest, field_set.output_path.value))
output_path = field_set.output_path.value_or_default(file_ending=None)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug fix: This code should have been calling .value_or_default.

@@ -40,16 +41,23 @@ class PyOxidizerOutputPathField(OutputPathField):
)


class PyOxidizerBinaryNameField(OutputPathField):
class PyOxidizerBinaryNameField(StringField, AsyncFieldMixin):
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This field should not have been subclassing OutputPathField since the field PyOxidizerOutputPathField is already a subclass of OutputPathField.

Comment on lines 116 to 118
normalized_spec_path=normalized_spec_path,
normalized_address=normalized_address,
file_suffix=file_suffix,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any one: Please propose better names for these.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Documenting this is more important than bikeshedding the names... Maybe add docs somewhere in a followup?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure. Any suggestions as to location? We have several references to output_path in the docs, but most of the form "You can optionally set the field output_path to change [the generated output name]" and nothing pants package specific from what I can see.

@tdyas tdyas changed the title [WIP] use template for output_path use a template for computing output_path for packageable targets Jul 17, 2024
@tdyas tdyas marked this pull request as ready for review July 17, 2024 16:04
@tdyas tdyas requested review from benjyw and thejcannon July 17, 2024 16:04
Copy link
Contributor

@benjyw benjyw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The names seem fine to me!

@tdyas
Copy link
Contributor Author

tdyas commented Jul 17, 2024

The names seem fine to me!

@benjyw: Any feedback on the choice of template engine? We seem to have three now:

  • Config files: %(foo)s
  • Chroot replacement: {chroot}
  • And this PR (via string.Template): $foo / ${foo}

@tdyas tdyas merged commit 418b837 into pantsbuild:main Jul 17, 2024
25 checks passed
@tdyas tdyas deleted the interpolate_artifact_name branch July 17, 2024 18:55
@benjyw
Copy link
Contributor

benjyw commented Jul 17, 2024

The names seem fine to me!

@benjyw: Any feedback on the choice of template engine? We seem to have three now:

  • Config files: %(foo)s
  • Chroot replacement: {chroot}
  • And this PR (via string.Template): $foo / ${foo}

Hmm, good question. The config file templating is really ini-specific (and we inherited it into our toml format). I think the string.Template thing is pretty robust, but is it supported in other languages? What if we replaced this code with Rust in the future?

tobni pushed a commit to tobni/pants that referenced this pull request Jul 22, 2024
…antsbuild#21175)

As described in pantsbuild#18659, when a
user sets `output_path` on a packageable target, the user gives up the
default behavior which uses a normalized version of the target's
directory as the directory into which the place the built artifact.
While the user could replicate the behavior manually, it would be a
better user experience if the user can still have partial access to some
of the default behavior.

This PR teaches the `output_path` field how to derive its value from a
template instead of hard-coding its behavior in Python. The
`output_path` field learns the following template replacements:

- `spec_path_normalized`: The path to the target's directory ("spec
path") with forward slashes replaced by dots.
- `target_name_normalized`: The target's name 
- `file_suffix`: For target's which produce single file artifacts, this
is the file type suffix to use. This is empty if not applicable.

The default value for the `output_path` field is now
`"${spec_path_normalized}/${target_name_normalized}${file_suffix}"`
which replicates the existing behavior.

`string.Template` is used as the template engine.

This PR also fixes some incorrect usages of `output_path` in several
backends.
huonw added a commit to pantsbuild/pantsbuild.org that referenced this pull request Jul 25, 2024
This escapes `$` in default values to avoid them being treated as
template literal substitutions.

Default values end up in a template literal `` `something` ``. If the
string contains `${...}` literally, they'll be treated as a
substitution. This PR escapes all `$`s to avoid this.

This bug/oversight doesn't cause problems with any _current_ values, but
will be required in 2.23.0.dev4 due to the new default value for
`output_path` fields from
pantsbuild/pants#21175.

Example of the effect (from
#234 (comment)):

```diff
- default_repr={`'${spec_path_normalized}/${target_name_normalized}${file_suffix}'`}
+ default_repr={`'\${spec_path_normalized}/\${target_name_normalized}\${file_suffix}'`}
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants