[Feature] Expose the step API to fixtures

[dvoytenko]

Our fixture performs some advanced mocking and it would be very convenient if it could output information, similar to how "route" information is output as steps. The information that could be displayed for a mocking operation would be:

• Call info: API name, API parameters
• Call result
• Additional log messages and error status

We're using the test.step() API right now, but it doesn't allow setting most of these parameters. We've also tried to use the private _runAsStep(), which allows setting all the data, but not a public API.

[dgozman]

@dvoytenko Could you share a short snippet illustrating your usage of _runAsStep()? Perhaps you can put all the information into a step title and call test.step() instead?

[dvoytenko]

@dgozman

@dvoytenko Could you share a short snippet illustrating your usage of _runAsStep()? Perhaps you can put all the information into a step title and call test.step() instead?

Here's an example from Playwright UI of what we're doing in our fixture:

And here's an example of a failed mock:

The only way we found how to do this is by using the _runAsStep internal API:

await testInfo._runAsStep(
  {
    category: 'mock',
    apiName: `mock.fulfilled`,
      params: {
        service: serviceName,
        ...serviceArgs, 
  },
  async (step) => {
    if (serviceFailure) {
       step.complete({ error: { message: serviceFailure } });
    }
  },
);

We'd definitely use test.step if it allowed us to add all this metadata and the error status.

An aside on the error status. Ideally, a step can be marked as failing without interrupting the complete test. Currently, if the test.step callback fails - the test is fully interrupted. Hence the use of step.complete({ error }) above.

[dgozman]

@dvoytenko We can probably add more options to test.step() like key-value parameters. Sounds like a nice feature request. I'll leave this issue open for prioritization.

Ideally, a step can be marked as failing without interrupting the complete test

You should be able to catch the step:

await test.step('title', async () => {
  throw new Error();
}).catch(() => {});

[dvoytenko]

Thanks! I think this would be a great feature.

[dvoytenko]

@dgozman I also noticed that there's a "LOGS" section inside the "Call" tab. Not critical, but if that could also be exposed - that'd help some use cases.