VLLM Compatibility Update

[wongjingping]

Changes

• update api_server to be compatible with latest version of vllm
• removed deprecated param cot_table_alias from run_model_cot.sh

Testing

Tested with:

./run_model_cot.sh defog/sqlcoder-8b-dare-ties

Results changed slightly due to running on an A40 with more memory, but overall still similar to past runs:

Model	sqlcoder-8b-dare-ties
V1 Accuracy	88.10%
Basic Accuracy	100%
Advanced Accuracy	90.62%
Overall Accuracy	90.1%

[Muhammad18557]

Cool, thanks! Will also pin the version in requirements.txt if you think this post1 version will be the standard. Or can keep as it is for backward compatibility!

[wongjingping]

Cool, thanks! Will also pin the version in requirements.txt if you think this post1 version will be the standard. Or can keep as it is for backward compatibility!

Thanks for the suggestion! We have some environments (e.g. the lambda labs H100) where it might not be convenient to upgrade the version due to existing hosted endpoints/models that we would need to test for regressions, hence will not pin the vllm version for now. 😄

[wongjingping]

Ok so after a lot of experimentation it turns out that spacy==3.7 is required for the tests to work as before