LLM: Refactor Pipeline-Parallel-FastAPI example #6636

Job	Run time
setup-python-version	1s
llm-cpp-build / check-linux-amx-artifact	3s
llm-cpp-build / check-linux-avx512-artifact	2s
llm-cpp-build / check-windows-avx2-artifact	3s
llm-cpp-build / check-linux-avxvnni-artifact	3s
llm-cpp-build / check-windows-avx-artifact	2s
llm-cpp-build / check-windows-avx-vnni-artifact	4m 18s
llm-cpp-build / windows-build-avx	2m 2s
llm-cpp-build / windows-build-avx2	2m 2s
llm-cpp-build / linux-build-avx512	2m 27s
llm-cpp-build / linux-build-avxvnni	1m 19s
llm-cpp-build / linux-build-amx	2m 10s
llm-cpp-build / windows-build-avx-vnni	2m 27s
llm-unit-test-on-arc (arc-ut, 2.1, 3.11)	24m 20s
llm-unit-test (windows, 3.11)	13m 16s
llm-unit-test-on-arc (arc-ut-win, 2.1, 3.11)	23m 50s
llm-unit-test (ubuntu-20.04-lts, 3.11)	15m 12s
	1h 33m 37s

Provide feedback