Skip to content

Commit

Permalink
Update llm_performance_tests.yml to include transformer 4.38.2
Browse files Browse the repository at this point in the history
  • Loading branch information
SichengStevenLi authored Aug 6, 2024
1 parent 687cbca commit 4a818a1
Showing 1 changed file with 32 additions and 0 deletions.
32 changes: 32 additions & 0 deletions .github/workflows/llm_performance_tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -213,6 +213,38 @@ jobs:
sed -i 's/batch2/batch4/g' run.py
python run.py
mv *.csv test_batch4
- name: Test on xpu(transformers==4.38.2)
shell: bash
run: |
source /opt/intel/oneapi/setvars.sh
export USE_XETLA=OFF
export SYCL_PI_LEVEL_ZERO_USE_IMMEDIATE_COMMANDLISTS=1
# upgrade transformers for model stablelm/stablelm-zephyr-3b & Gemma/gemma-7b-it
python -m pip install transformers==4.38.2
# batch_size 1
cp python/llm/test/benchmark/arc-perf-transformers-438.yaml python/llm/dev/benchmark/all-in-one/config.yaml
cd python/llm/dev/benchmark/all-in-one
# change csv name
sed -i 's/test1_batch4/test2_batch1/g' run.py
python run.py
mv *.csv test_batch1
# batch_size 2
cd ../../../../../
cp python/llm/test/benchmark/arc-perf-transformers-438-batch2.yaml python/llm/dev/benchmark/all-in-one/config.yaml
cd python/llm/dev/benchmark/all-in-one
# change csv name
sed -i 's/batch1/batch2/g' run.py
python run.py
mv *.csv test_batch2
# batch_size 4
cd ../../../../../
cp python/llm/test/benchmark/arc-perf-transformers-438-batch4.yaml python/llm/dev/benchmark/all-in-one/config.yaml
cd python/llm/dev/benchmark/all-in-one
# change csv name
sed -i 's/batch2/batch4/g' run.py
python run.py
mv *.csv test_batch4
- name: Test on xpu(transformers==4.40.0)
shell: bash
Expand Down

0 comments on commit 4a818a1

Please sign in to comment.