Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test common workflows across different instance types, cluster sizes #241

Open
ian-r-rose opened this issue Aug 12, 2022 · 1 comment
Open

Comments

@ian-r-rose
Copy link
Contributor

If a workflow is struggling, there are often a number of plausible fixes, including:

  • Changing chunk/partition sizes
  • Increasing cluster size
  • Choosing a bigger instance type.

We don't always have strong guidance for users about what to choose in this case. It would be good to include some tests parameterized across these dimensions to better inform coiled users. I know the @gjoseph92 has some work-in-progress towards that end, and @ntabris has expressed interest.

@ntabris
Copy link
Member

ntabris commented Aug 12, 2022

From my perspective, I'd love for the platform to be able to tell people how to save money running their workloads, and this data would potentially enable some of that (e.g., we can tell you taht your workload would finish faster if you used fewer, larger instances).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants