Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test(instr-openai): refactor some test envvars and the Q used for testing chat completion #497

Merged
merged 3 commits into from
Dec 26, 2024
Merged

test(instr-openai): refactor some test envvars and the Q used for testing chat completion #497

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
dc3f17a
test(instr-openai): refactor test envvar names
trentm Dec 19, 2024
44ce8fa
refactor: update the chat Q used in tests
trentm Dec 19, 2024
c9db60d
update exact values for updated pre-recorded responses
trentm Dec 19, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/test-instrumentation-openai.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -78,8 +78,8 @@ jobs:
- name: Pull Ollama models
run: |
source packages/instrumentation-openai/ollama.env
curl -s http://localhost:11434/api/pull -d "{\"model\": \"$TEST_MODEL_TOOLS\"}"
curl -s http://localhost:11434/api/pull -d "{\"model\": \"$TEST_MODEL_EMBEDDINGS\"}"
curl -s http://localhost:11434/api/pull -d "{\"model\": \"$TEST_CHAT_MODEL\"}"
curl -s http://localhost:11434/api/pull -d "{\"model\": \"$TEST_EMBEDDINGS_MODEL\"}"
curl -s http://localhost:11434/api/tags | jq
# Dump Ollama container logs if it doesn't appear to be working.
curl -fsS http://localhost:11434/ || docker logs $(docker ps -q)
Expand Down
2 changes: 1 addition & 1 deletion packages/instrumentation-openai/CONTRIBUTING.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ dev/debugging. For example:
npx @elastic/mockotlpserver # or whatever OTLP endpoint you like to use

cd test/fixtures
TEST_MODEL_TOOLS=gpt-4o-mini \
TEST_CHAT_MODEL=gpt-4o-mini \
node --env-file ../../openai.env -r ./telemetry.js chat-completion.js
```

4 changes: 2 additions & 2 deletions packages/instrumentation-openai/TESTING.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,8 +57,8 @@ or a running Ollama. Which is used is determined by the environment variables.
docker run -it --rm -p 11434:11434 -v ~/.ollama:/root/.ollama ghcr.io/elastic/ollama/ollama:testing serve

set -a; source ./ollama.env
ollama pull $TEST_MODEL_TOOLS
ollama pull $TEST_MODEL_EMBEDDINGS
ollama pull $TEST_CHAT_MODEL
ollama pull $TEST_EMBEDDINGS_MODEL

npm run test:integration
```
Expand Down
4 changes: 2 additions & 2 deletions packages/instrumentation-openai/azure.env.template
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,10 +28,10 @@ OPENAI_API_VERSION=2024-10-01-preview

# Set this to the *deployment name* in your Azure OpenAI endpoint that deployed
# a model that supports OpenAI tool calling, such as "gpt-4o-mini".
TEST_MODEL_TOOLS=YOUR_TOOL_COMPATIBLE_DEPLOYMENT_NAME
TEST_CHAT_MODEL=YOUR_TOOL_COMPATIBLE_DEPLOYMENT_NAME

# Set this to the *deployment name* in your Azure OpenAI endpoint that deployed
# a model that supports OpenAI embeddings, such as "text-embedding-3-small".
TEST_MODEL_EMBEDDINGS=YOUR_EMBEDDING_DEPLOYMENT_NAME
TEST_EMBEDDINGS_MODEL=YOUR_EMBEDDING_DEPLOYMENT_NAME


2 changes: 1 addition & 1 deletion packages/instrumentation-openai/examples/use-chat-esm.mjs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ async function main() {
{
role: 'user',
content:
'Answer in up to 3 words: Which ocean contains the falkland islands?',
'Answer in up to 3 words: Which ocean contains Bouvet Island?',
},
],
});
Expand Down
2 changes: 1 addition & 1 deletion packages/instrumentation-openai/examples/use-chat.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ async function main() {
{
role: 'user',
content:
'Answer in up to 3 words: Which ocean contains the falkland islands?',
'Answer in up to 3 words: Which ocean contains Bouvet Island?',
},
],
});
Expand Down
4 changes: 2 additions & 2 deletions packages/instrumentation-openai/ollama.env
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,5 +4,5 @@ OPENAI_API_KEY=notused

# These models may be substituted in the future with inexpensive to run, newer
# variants.
TEST_MODEL_TOOLS=qwen2.5:0.5b
TEST_MODEL_EMBEDDINGS=all-minilm:33m
TEST_CHAT_MODEL=qwen2.5:0.5b
TEST_EMBEDDINGS_MODEL=all-minilm:33m
4 changes: 2 additions & 2 deletions packages/instrumentation-openai/openai.env.template
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,5 +20,5 @@ OPENAI_API_KEY=sk-...
# OPENAI_ORG_ID=org-...
# OPENAI_PROJECT_ID=...

TEST_MODEL_TOOLS=gpt-4o-mini
TEST_MODEL_EMBEDDINGS=text-embedding-3-small
TEST_CHAT_MODEL=gpt-4o-mini
TEST_EMBEDDINGS_MODEL=text-embedding-3-small
Loading
Loading