Allow several --parameter-list flags to set

[ghost]

Description

Right now, we can use --parameter-list and that's great. I like the idea of parametrised benchmarks. Though, sometimes, you need to specify several parameter lists with different values.

Expected Behaviour

When calling hyperfine:

hyperfine \
  --parameter-list npm_flags --some,--flags,--here \
  --parameter-list yarn_flags --some,--other,--flags \
  "npm install {npm_flags}" \
  "yarn install {yarn_flags}"

I expect to have a possibility to parametrise different clients, that expect different flags. Surely, I can do it manually by duplicating commands for each flag, but why when you have such great flag.

Current Behaviour

When calling the command like above, I'm getting:

error: The following required arguments were not provided:
    <command>...

USAGE:
    hyperfine <command>... --parameter-list <VAR> <VALUES> --prepare <CMD>...

For more information try --help

Notes

I had a look into your code and seems like you does not support --parameter-list flag as collection, but as single possible value.

Is there any way to implement such kind of behaviour?

[sharkdp]

Thank you for the feedback.

If we would implement this, it would interfere with what is requested in #253. For me, it seems like #253 is the more common use case, as compared to this one. I think this ticket would actually be solved by the "zipped" version discussed in #253?

[ghost]

#253 is what I want to have, actually, it does not interfere 😄

I see that you still not sure about syntax and flags for it, but semantically that's exactly I want to achieve - a way to create different variables and assign to them different values, so I could expand it while benchmarking.

For now, I made a workaround as an array of commands to run in Bash script:

SCRIPTS=(
    "${NPM_INSTALL} && ${NPM_BUILD}"
    "${NPM_INSTALL} --prefer-offline && ${NPM_BUILD}"
    "${NPM_INSTALL} --prefer-online && ${NPM_BUILD}"
    "${YARN_INSTALL} && ${YARN_BUILD}"
    "${PNPM_INSTALL} && ${PNPM_BUILD}"
    "${PNPM_INSTALL} --prefer-offline && ${PNPM_BUILD}"
)

hyperfine \
    --prepare "${CLEAN_ARTIFACTS} && ${CLEAN_CACHE}" \
    --export-json cold_cache_without_node_modules.json \
    "${SCRIPTS[@]}"

[sharkdp]

closing this in favor of #253