ci: Add ability to array-ify args and run multiple jobs

[raunakab]

Overview

Previously, the run-cluster workflow only ran one ray-job-submission. This PR extends the ability to be able to run any arbitrary array of job submissions by enabling us to pass an array into the entrypoint_args input param. This then splits the command into its multiple pieces and submits them all.

Example Usage

gh workflow run run-cluster.yaml \
    --ref $current_branch \
    -f working_dir="." \
    -f daft_wheel_url="https://github-actions-artifacts-bucket.s3.us-west-2.amazonaws.com/builds/54428e3738e96764af60cfdd8a0e4a41717ec9f9/getdaft-0.3.0.dev0-cp38-abi3-manylinux_2_31_x86_64.whl" \
    -f entrypoint_script="benchmarking/tpcds/ray_entrypoint.py" \
    -f entrypoint_args="[\"--tpcds-gen-folder='gendata' --question='1'\", \"--tpcds-gen-folder='gendata' --question='2'\"]"

The above invocation runs TPC-DS queries 1 and 2.

[raunakab]

There are some hardcoded values in the runner script (that is run on the runner node, not the ray-head node).

However, we don't have a mechanism to pass values along to the runner script. I'm thinking of creating one named runner_scripts_args. Thoughts @jaychia?

[codspeed-hq]

CodSpeed Performance Report

Merging #3584 will degrade performances by 37.6%

_{Comparing ci/run-cluster (5b5a9f9) with main (e148248)}

Summary

❌ 1 regressions
✅ 26 untouched benchmarks

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Benchmarks breakdown

	Benchmark	main	ci/run-cluster	Change
❌	test_iter_rows_first_row[100 Small Files]	142.9 ms	229.1 ms	-37.6%

[jaychia]

Should not be generating TPC-DS data for every run-cluster command.

Not sure what's going on there, also you're generating this on the runner, but we need the data to be available in S3?

[raunakab]

Example of a successful run:
https://github.com/Eventual-Inc/Daft/actions/runs/12361707341

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 77.82%. Comparing base (6c21917) to head (ff89642).
Report is 3 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3584      +/-   ##
==========================================
+ Coverage   77.79%   77.82%   +0.02%     
==========================================
  Files         716      716              
  Lines       87991    88243     +252     
==========================================
+ Hits        68455    68673     +218     
- Misses      19536    19570      +34     

see 3 files with indirect coverage changes

[jaychia]

Can we point to a successful job run as well?

[raunakab]

Example of a successful run (run on just 1 argument, --question=3 --scale-factor=100):
https://github.com/Eventual-Inc/Daft/actions/runs/12366475502

[raunakab]

Example of a successful run (run on multiple arguments, ["--question=1 --scale-factor=100", "--question=3 --scale-factor=100"]):
https://github.com/Eventual-Inc/Daft/actions/runs/12380369256