Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor to remove test args from remote test command #3266

Merged
merged 1 commit into from
Oct 14, 2024

Conversation

tm-jdelapuente
Copy link
Contributor

@tm-jdelapuente tm-jdelapuente commented Oct 7, 2024

Context

plz test [TARGET] can take an argument to specify specific parts of a test target to run.

For example, plz test //[TARGET] -- [TestSubTarget] runs the subtarget function rather than all of the test functions in the target.

You can also specify what tests to run at a more granular level: e.g. [TestSubTarget/[NestedTestFunction]/[NestedTestFunction]

Problem
When specifying subtargets in plz test ... you get an error when you pass an argument that has a slash (e.g. plz test //[TARGET] -- TestSubTarget/MyTestFunction.

This shows up by running an additional test that errors, such as

tee: TestSubTarget/TestMyTestFunction/BulkGetMyResources: No such file or directory

Root cause

Please is constructing a cmd that calls tee and appends TestArgs at the end.

cmd, err := core.ReplaceTestSequences(state, target, target.GetTestCommand(state))
if len(state.TestArgs) != 0 {
	cmd += " " + strings.Join(state.TestArgs, " ")
}

This constructs a command like $TEST 2>&1 | tee $TMP_DIR/test.results [one or more subtests].

This is relevant because tee reads from stdin ($TEST 2>&1) and writes to stdout and files ($TMP_DIR/test.results [one or more subtests]).

Thus, this means that

  • plz test //[TARGET] -- [TestSubTarget] creates and writes to the file TestSubTarget
  • plz test //[TARGET] -- [TestSubTarget/[NestedTestFunction]/[NestedTestFunction] interprets the forward slashes as file paths and tries to create a file in said path but errors because the directory doesn't exist

Solution
The implication is that we need to avoid passing the subtests before the pipe to tee.

I thus removed the if statement to avoid passing the test arguments in the command.

Testing
I built Please with these changes and it worked as expected, only printing the subtests I passed and no erroring like it does now.

I'm concerned about any regressions and am not sure how to best test this.

Pending work
This only solves the problem on remote execution, not local execution.

@tm-jdelapuente tm-jdelapuente changed the title Refactor to pass subtest before the pipe [WIP] Refactor to pass subtest before the pipe Oct 7, 2024
@tm-jdelapuente tm-jdelapuente changed the title [WIP] Refactor to pass subtest before the pipe [WIP] Refactor to remove test args from test command Oct 9, 2024
@tm-jdelapuente tm-jdelapuente changed the title [WIP] Refactor to remove test args from test command [WIP] Refactor to remove test args from remote test command Oct 9, 2024
@tm-jdelapuente tm-jdelapuente changed the title [WIP] Refactor to remove test args from remote test command [Refactor to remove test args from remote test command Oct 9, 2024
@tm-jdelapuente tm-jdelapuente changed the title [Refactor to remove test args from remote test command Refactor to remove test args from remote test command Oct 9, 2024
@tm-jdelapuente tm-jdelapuente marked this pull request as ready for review October 9, 2024 15:09
@peterebden
Copy link
Member

I think we pass this as an environment variable now, yes?

@tm-jdelapuente
Copy link
Contributor Author

What do you mean? I don't think I'm following, because the above seems unrelated to that since it's about tee returning an error because it tries to write to a directory that doesn't exist

@peterebden
Copy link
Member

The information about which sub-tests to run must be given to it somehow. IIRC it was originally an argument but then we changed it to an env var to avoid issues like this. I'm checking whether that does in fact happen as expected.

@tm-jdelapuente
Copy link
Contributor Author

Just double checking, I understand you don't expect any further action from me while you check it?

@peterebden
Copy link
Member

Yup ok we set one called TESTS which is read by the various test runners.
(In the case of the Go one at least, it isn't looking at arguments for it, you'd have to add -test.run as well for that to work)

@tm-jdelapuente tm-jdelapuente merged commit 2819f51 into master Oct 14, 2024
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants