[CI/Build] Split up models tests

[DarkLight1337]

To reduce the impact of flakiness (e.g. connection failures), this PR splits up the model tests into groups.

I have also fixed VLM test failures that were introduced by #8346 and #9983.

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.
Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can do one of these:

• Add ready label to the PR
• Enable auto-merge.

🚀

[DarkLight1337]

It looks like sharding doesn't work over parameter combinations.

[DarkLight1337]

Would be great if we could get this in before the next nightly!

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @DarkLight1337.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[youkaichao]

thanks for the fix!

[youkaichao]

I merged it to fix the test on main ASAP. also left some comments.

[youkaichao]

it would be better if this can use init_vllm_registered_model .

I think this config.text_config misses an architectures field. we can manually add the field, and then call init_vllm_registered_model.

we also need to change the signature of init_vllm_registered_model in the future.

[DarkLight1337]

It has model_type defined so it should still be possible to match if we add this model to the registry.

[youkaichao]

we should flatten VllmConfig in the future, hf_config can be the same level as model_config . and we can have a top-level function vllm_config.replace_with(hf_config=hf_config)

[DarkLight1337]

Some values in model_config depend on the hf_config, so it might be better to just have a convenience property to access hf_config directly from VllmConfig, instead of changing the nested structure.