Python: Add OpenTelemetry to Python SK #6914

glahaye · 2024-06-23T23:12:43Z

Motivation and Context

We want observability into usage of SK

Description

Add OpenTelemetry to Python SK

Contribution Checklist

The code builds clean without any errors or warnings
The PR follows the SK Contribution Guidelines and the pre-submission formatting script raises no violations

markwallace-microsoft · 2024-06-23T23:13:17Z

Python 3.10 Test Coverage Report

File	Stmts	Miss	Cover	Missing
semantic_kernel
kernel.py	157	9	94%	145, 156, 160, 200–201, 310–313, 422
semantic_kernel/connectors/ai/azure_ai_inference/services
azure_ai_inference_chat_completion.py	149	4	97%	169, 312, 416, 430
semantic_kernel/connectors/ai/embeddings
embedding_generator_base.py	8	1	88%	50
semantic_kernel/connectors/ai/hugging_face/services
hf_text_completion.py	54	3	94%	95, 103, 134
hf_text_embedding.py	32	5	84%	77–81
semantic_kernel/connectors/openapi_plugin
openapi_manager.py	58	2	97%	110–111
openapi_parser.py	88	1	99%	127
openapi_runner.py	105	2	98%	181–182
semantic_kernel/connectors/openapi_plugin/models
rest_api_operation.py	129	1	99%	240
semantic_kernel/functions
kernel_function_decorator.py	98	1	99%	102
kernel_function_from_method.py	96	1	99%	153
kernel_function_from_prompt.py	154	7	95%	165–166, 180, 201, 219, 239, 322
kernel_plugin.py	187	2	99%	472, 475
semantic_kernel/planners
plan.py	234	45	81%	52, 161–163, 195, 212–225, 262, 267, 275–276, 286–289, 306, 311, 327, 330–335, 353, 358, 361, 363, 370, 384–386, 391–395
semantic_kernel/planners/function_calling_stepwise_planner
function_calling_stepwise_planner.py	116	4	97%	143, 187–188, 196
semantic_kernel/planners/sequential_planner
sequential_planner.py	64	6	91%	69, 73, 107, 123, 132–133
sequential_planner_extensions.py	50	9	82%	29–30, 52, 106–120
sequential_planner_parser.py	77	12	84%	64–72, 91, 115–118
semantic_kernel/schema
kernel_json_schema_builder.py	119	5	96%	48, 82, 204, 208–209
semantic_kernel/template_engine/blocks
code_block.py	77	1	99%	119
named_arg_block.py	43	1	98%	98
semantic_kernel/utils/telemetry
decorators.py	113	4	96%	232, 240, 243, 252
user_agent.py	16	2	88%	18–19
TOTAL	7471	128	98%

Python 3.10 Unit Test Overview

Tests	Skipped	Failures	Errors	Time
1940	1 💤	0 ❌	0 🔥	33.706s ⏱️

python/poetry.lock

eavanvalkenburg

Some work to do, the basics are there, but most important is to refactor this into a decorator that we can use with all completion services and functions!

python/semantic_kernel/connectors/ai/open_ai/services/open_ai_chat_completion_base.py

python/semantic_kernel/utils/model_diagnostics.py

python/semantic_kernel/connectors/ai/open_ai/services/open_ai_chat_completion_base.py

python/semantic_kernel/utils/model_diagnostics.py

python/semantic_kernel/connectors/ai/open_ai/services/open_ai_chat_completion_base.py

python/semantic_kernel/utils/model_diagnostics.py

python/semantic_kernel/connectors/ai/open_ai/services/open_ai_chat_completion_base.py

…chat_completion_base.py Co-authored-by: Tao Chen <[email protected]>

python/semantic_kernel/connectors/ai/open_ai/services/open_ai_chat_completion_base.py

eavanvalkenburg

I see both naming with diagnostics (env setting and model_diagnostics.py file) and tracing, let's make that more consistent! And we really need to create a decorator out of the code inside openai_chat_completion_base!

python/semantic_kernel/connectors/ai/open_ai/services/open_ai_chat_completion_base.py

python/semantic_kernel/utils/model_diagnostics.py

…c-kernel into python_telemetry

eavanvalkenburg

couple of small notes, but most importantly I don't see any tests for the actual span stuff...

python/semantic_kernel/utils/tracing/decorators.py

moonbox3 · 2024-07-23T13:32:28Z

Related to #5891

eavanvalkenburg

couple of small notes, starting to look good!

python/semantic_kernel/utils/tracing/__init__.py

python/semantic_kernel/utils/tracing/const.py

eavanvalkenburg · 2024-07-24T09:11:35Z

Also just noticed that there are a bunch of lines in decorators.py not covered by tests, have a look at those please, @glahaye

glahaye · 2024-07-24T18:21:42Z

Also just noticed that there are a bunch of lines in decorators.py not covered by tests, have a look at those please, @glahaye

Oh, yes, I didn't know I needed "@pytest.mark.asyncio" for async tests.

Fixed.

…c-kernel into python_telemetry

python/pyproject.toml

python/semantic_kernel/connectors/ai/open_ai/services/open_ai_chat_completion_base.py

### Motivation and Context We want observability into usage of SK ### Description Add OpenTelemetry to Python SK ### Contribution Checklist - [ ] The code builds clean without any errors or warnings - [ ] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations --------- Co-authored-by: Tao Chen <[email protected]>

loggin

22204aa

glahaye requested a review from a team as a code owner June 23, 2024 23:12

markwallace-microsoft added the python Pull requests for the Python Semantic Kernel label Jun 23, 2024

github-actions bot changed the title ~~Add OpenTelemetry to Python SK~~ Python: Add OpenTelemetry to Python SK Jun 23, 2024

Merge main

1156b46

glahaye commented Jun 24, 2024

View reviewed changes

python/poetry.lock Outdated Show resolved Hide resolved

eavanvalkenburg requested changes Jun 24, 2024

View reviewed changes

Merge branch 'main' into python_telemetry

e5ed784

TaoChenOSU reviewed Jun 24, 2024

View reviewed changes

alliscode reviewed Jun 24, 2024

View reviewed changes

python/semantic_kernel/connectors/ai/open_ai/services/open_ai_chat_completion_base.py Outdated Show resolved Hide resolved

glahaye and others added 3 commits June 24, 2024 11:45

Update python/semantic_kernel/connectors/ai/open_ai/services/open_ai_…

a2c5fd7

…chat_completion_base.py Co-authored-by: Tao Chen <[email protected]>

Addressed PR issues

2848051

Merge main

fe86266

glahaye requested review from TaoChenOSU, eavanvalkenburg and alliscode June 27, 2024 21:46

glahaye added 2 commits June 27, 2024 14:56

Merge main

e9dd311

Update poetry info after merge

4f3783d

alliscode reviewed Jun 27, 2024

View reviewed changes

python/semantic_kernel/connectors/ai/open_ai/services/open_ai_chat_completion_base.py Outdated Show resolved Hide resolved

alliscode reviewed Jun 27, 2024

View reviewed changes

python/semantic_kernel/connectors/ai/open_ai/services/open_ai_chat_completion_base.py Outdated Show resolved Hide resolved

alliscode reviewed Jun 27, 2024

View reviewed changes

python/semantic_kernel/connectors/ai/open_ai/services/open_ai_chat_completion_base.py Outdated Show resolved Hide resolved

glahaye added 3 commits June 27, 2024 16:03

Address PR issues

e71fab2

Fix lint warning

6bf6406

Fix false positive security issue

00269b9

eavanvalkenburg requested changes Jun 28, 2024

View reviewed changes

glahaye added 4 commits July 8, 2024 11:29

completion telemetry now in decorator

1fac03f

Use classvar

cc13f52

Merge main

6437b04

Proper merge of poetry lock

c6aa839

glahaye added 5 commits July 15, 2024 14:54

Use kwargs for prompt, chat_history and settings

d95c8d2

Merge branch 'main' into python_telemetry

76739cb

Merge main

e0a2471

Merge branch 'python_telemetry' of https://github.com/glahaye/semanti…

a04bbd6

…c-kernel into python_telemetry

Fix poetry.lock after merge from main

17c2c72

eavanvalkenburg reviewed Jul 18, 2024

View reviewed changes

python/semantic_kernel/utils/tracing/decorators.py Outdated Show resolved Hide resolved

python/semantic_kernel/utils/tracing/decorators.py Outdated Show resolved Hide resolved

python/semantic_kernel/utils/tracing/decorators.py Outdated Show resolved Hide resolved

glahaye added 4 commits July 23, 2024 13:42

Add unit tests

1f163f1

Merge main

6710051

Adapt poetry.lock after merge

d73ead7

Merge branch 'main' into python_telemetry

066e370

eavanvalkenburg reviewed Jul 24, 2024

View reviewed changes

glahaye added 3 commits July 24, 2024 11:28

Address PR comments + enable async unit tests

6ae2ef6

Merge branch 'python_telemetry' of https://github.com/glahaye/semanti…

3e8d5e9

…c-kernel into python_telemetry

Fix unit tests

c001002

markwallace-microsoft added the memory label Jul 24, 2024

glahaye added 4 commits July 24, 2024 14:44

Overriding .env file in unit tests

1a73614

Fix unit tests

dabe4c2

Merge main

32e03bb

Fix poetry.lock after merge

f2b0f5d

eavanvalkenburg approved these changes Jul 25, 2024

View reviewed changes

moonbox3 approved these changes Jul 25, 2024

View reviewed changes

python/pyproject.toml Show resolved Hide resolved

python/semantic_kernel/connectors/ai/open_ai/services/open_ai_chat_completion_base.py Show resolved Hide resolved

glahaye enabled auto-merge July 25, 2024 20:18

glahaye added this pull request to the merge queue Jul 25, 2024

Merged via the queue into microsoft:main with commit ca78ff7 Jul 25, 2024
24 checks passed

glahaye deleted the python_telemetry branch July 25, 2024 20:40

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Python: Add OpenTelemetry to Python SK #6914

Python: Add OpenTelemetry to Python SK #6914

glahaye commented Jun 23, 2024

markwallace-microsoft commented Jun 23, 2024 •

edited

Loading

eavanvalkenburg left a comment

eavanvalkenburg left a comment

eavanvalkenburg left a comment

moonbox3 commented Jul 23, 2024

eavanvalkenburg left a comment

eavanvalkenburg commented Jul 24, 2024

glahaye commented Jul 24, 2024

Python: Add OpenTelemetry to Python SK #6914

Python: Add OpenTelemetry to Python SK #6914

Conversation

glahaye commented Jun 23, 2024

Motivation and Context

Description

Contribution Checklist

markwallace-microsoft commented Jun 23, 2024 • edited Loading

Python 3.10 Unit Test Overview

eavanvalkenburg left a comment

Choose a reason for hiding this comment

eavanvalkenburg left a comment

Choose a reason for hiding this comment

eavanvalkenburg left a comment

Choose a reason for hiding this comment

moonbox3 commented Jul 23, 2024

eavanvalkenburg left a comment

Choose a reason for hiding this comment

eavanvalkenburg commented Jul 24, 2024

glahaye commented Jul 24, 2024

markwallace-microsoft commented Jun 23, 2024 •

edited

Loading