feat: add bedrock client

[takipipo]

Feature

• Add bedrock client

[takipipo]

Still not sure please confirm

[mhokchuekchuek]

ถ้าในกรณีที่ไม่ได้ add request_config.sampling_params ตอนรัน command
มันจะถูก set ว่า request_config.sampling_params = {"max_tokens": num_output_tokens}
ref:

llmperf/token_benchmark_ray.py

Lines 67 to 69 in 7512357

	if not additional_sampling_params:
	additional_sampling_params = {}

llmperf/token_benchmark_ray.py

Lines 92 to 93 in 7512357

	default_sampling_params = {"max_tokens": num_output_tokens}
	default_sampling_params.update(additional_sampling_params)

llmperf/token_benchmark_ray.py

Lines 147 to 159 in 7512357

	metadata = {
	"model": model,
	"mean_input_tokens": mean_input_tokens,
	"stddev_input_tokens": stddev_input_tokens,
	"mean_output_tokens": mean_output_tokens,
	"stddev_output_tokens": stddev_output_tokens,
	"num_concurrent_requests": num_concurrent_requests,
	"additional_sampling_params": additional_sampling_params,
	}
	metadata["results"] = ret
	return metadata, completed_requests

soln:
แก้ "max_token" ให้เป็น parameter ที่ใช้บอก "max_token" ในแต่ละ client เช่น sagemaker ใช้ "max_new_tokens"

llmperf/src/llmperf/ray_clients/sagemaker_client.py

Lines 45 to 47 in 7512357

	if "max_tokens" in sampling_params:
	sampling_params["max_new_tokens"] = sampling_params["max_tokens"]
	del sampling_params["max_tokens"]