-
Notifications
You must be signed in to change notification settings - Fork 648
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Cloud Trace exporter #698
Cloud Trace exporter #698
Conversation
f789229
to
94c206f
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Quick initial pass on cloud_trace/init.py
ext/opentelemetry-ext-cloud-trace/src/opentelemetry/ext/cloud_trace/__init__.py
Outdated
Show resolved
Hide resolved
ext/opentelemetry-ext-cloud-trace/src/opentelemetry/ext/cloud_trace/__init__.py
Outdated
Show resolved
Hide resolved
ext/opentelemetry-ext-cloud-trace/src/opentelemetry/ext/cloud_trace/__init__.py
Outdated
Show resolved
Hide resolved
ext/opentelemetry-ext-cloud-trace/src/opentelemetry/ext/cloud_trace/__init__.py
Outdated
Show resolved
Hide resolved
ext/opentelemetry-ext-cloud-trace/src/opentelemetry/ext/cloud_trace/__init__.py
Outdated
Show resolved
Hide resolved
ext/opentelemetry-ext-cloud-trace/src/opentelemetry/ext/cloud_trace/__init__.py
Outdated
Show resolved
Hide resolved
ext/opentelemetry-ext-cloud-trace/src/opentelemetry/ext/cloud_trace/__init__.py
Outdated
Show resolved
Hide resolved
ext/opentelemetry-ext-cloud-trace/src/opentelemetry/ext/cloud_trace/__init__.py
Outdated
Show resolved
Hide resolved
ext/opentelemetry-ext-cloud-trace/src/opentelemetry/ext/cloud_trace/__init__.py
Outdated
Show resolved
Hide resolved
ext/opentelemetry-ext-cloud-trace/tests/test_cloud_trace_exporter.py
Outdated
Show resolved
Hide resolved
ext/opentelemetry-ext-cloud-trace/tests/test_cloud_trace_exporter.py
Outdated
Show resolved
Hide resolved
Quick notes from offline discussion:
|
ext/opentelemetry-ext-cloud-trace/src/opentelemetry/ext/cloud_trace/__init__.py
Outdated
Show resolved
Hide resolved
ext/opentelemetry-ext-cloud-trace/src/opentelemetry/ext/cloud_trace/__init__.py
Outdated
Show resolved
Hide resolved
b276f0a
to
124081b
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
One blocking comment about Span
-valued span parents, some questions about client types, and some non-blocking nits.
I'll have to take another pass at the examples, but this looks like a great start otherwise. Thanks for picking this up @AndrewAXue!
ext/opentelemetry-ext-cloud-trace/src/opentelemetry/ext/cloud_trace/version.py
Outdated
Show resolved
Hide resolved
ext/opentelemetry-ext-cloud-trace/src/opentelemetry/ext/cloud_trace/__init__.py
Outdated
Show resolved
Hide resolved
ext/opentelemetry-ext-cloud-trace/src/opentelemetry/ext/cloud_trace/__init__.py
Outdated
Show resolved
Hide resolved
ext/opentelemetry-ext-cloud-trace/src/opentelemetry/ext/cloud_trace/__init__.py
Outdated
Show resolved
Hide resolved
ext/opentelemetry-ext-cloud-trace/src/opentelemetry/ext/cloud_trace/__init__.py
Outdated
Show resolved
Hide resolved
ext/opentelemetry-ext-cloud-trace/src/opentelemetry/ext/cloud_trace/__init__.py
Outdated
Show resolved
Hide resolved
ext/opentelemetry-ext-cloud-trace/src/opentelemetry/ext/cloud_trace/__init__.py
Outdated
Show resolved
Hide resolved
ext/opentelemetry-ext-cloud-trace/src/opentelemetry/ext/cloud_trace/__init__.py
Outdated
Show resolved
Hide resolved
# pylint: disable=broad-except | ||
except Exception as ex: | ||
logger.warning("Error while writing to Cloud Trace: %s", ex) | ||
return SpanExportResult.FAILURE |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not specific to this PR but should SpanExportResult be able to encapsulate other error information like the original exception? If all exporters try to catch and swallow every exception globally and then return just one value to represent failure, we might be losing out on valuable information. Perhaps processor/exporter should be able to return the original python exceptions or error messages+stack traces along with a failure status. This information can be used by the caller to show helpful error messages to users by utilizing a single global error handler instead of relying on every processor to do the right thing by using the correct logger. In general, I feel error handling should propagate back up and be taken care of preferably in a single place. Especially if we are catching all possible exceptions.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree with you, but I would prefer to handle that in a seperate PR :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For sure, I'm not suggesting you change it in this PR but wanted to bring this up for the Python group in general.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree, although in this case hopefully the logger.warning helps.
One structural question: The majority of the code lives in init.py. This strikes me as an odd choice. Is there a reason for this, e.g. is there a style guide that prefers using init.py in certain cases? |
e0baf7a
to
25cdf3e
Compare
ext/opentelemetry-ext-cloud-trace/src/opentelemetry/ext/cloud_trace/__init__.py
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Before merging, we should create a minimal example, which can be manually run to sanity-check the exporter. I quickly tried to shoehorn this into an existing program, but got this error:
Error while writing to Cloud Trace: list indices must be integers or slices, not str
Error 'TruncatableString' object is not iterable when creating span
Oh, and for any issues, comments or suggestions that we decide not to address right now, please open an issue in https://github.com/GoogleCloudPlatform/opentelemetry-operations-python, and link it here. |
Hey Marius, there are examples in opentelemetry-ext-cloud-trace/examples. One of which is just a basic trace, the 2nd and 3rd are example server + clients. Can you send me the exact code you tried to use this exporter in? |
Taking the credentials/test environment discussion offline for debugging (I see similar issues with the examples in my env). |
cad4252
to
0a145fa
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi Andrew, I did a relatively thorough review, focusing mostly on the Trace API interaction.
Left a handful of comments.
As usual, feel free to defer comments to a later PRs upon creating appropriate GitHub issues.
ext/opentelemetry-ext-cloud-trace/src/opentelemetry/ext/cloud_trace/__init__.py
Outdated
Show resolved
Hide resolved
ext/opentelemetry-ext-cloud-trace/src/opentelemetry/ext/cloud_trace/__init__.py
Outdated
Show resolved
Hide resolved
ext/opentelemetry-ext-cloud-trace/src/opentelemetry/ext/cloud_trace/__init__.py
Outdated
Show resolved
Hide resolved
ext/opentelemetry-ext-cloud-trace/src/opentelemetry/ext/cloud_trace/__init__.py
Outdated
Show resolved
Hide resolved
ext/opentelemetry-ext-cloud-trace/src/opentelemetry/ext/cloud_trace/__init__.py
Outdated
Show resolved
Hide resolved
ext/opentelemetry-ext-cloud-trace/src/opentelemetry/ext/cloud_trace/__init__.py
Outdated
Show resolved
Hide resolved
ext/opentelemetry-ext-cloud-trace/src/opentelemetry/ext/cloud_trace/__init__.py
Outdated
Show resolved
Hide resolved
ext/opentelemetry-ext-cloud-trace/src/opentelemetry/ext/cloud_trace/__init__.py
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the PR! There's a few blocking comments for me. The major sticking point to me is the modification of a vendor-agnostic instrumentation specifically for cloud trace. Maybe we can discuss further here or through gitter on a solution there.
ext/opentelemetry-ext-cloud-trace/src/opentelemetry/ext/cloud_trace/__init__.py
Outdated
Show resolved
Hide resolved
ext/opentelemetry-ext-cloud-trace/src/opentelemetry/ext/cloud_trace/__init__.py
Outdated
Show resolved
Hide resolved
ext/opentelemetry-ext-cloud-trace/src/opentelemetry/ext/cloud_trace/__init__.py
Outdated
Show resolved
Hide resolved
ext/opentelemetry-ext-requests/src/opentelemetry/ext/requests/__init__.py
Outdated
Show resolved
Hide resolved
a131d1a
to
002279b
Compare
4b5d631
to
249beb4
Compare
@AndrewAXue FYI force pushing breaks github's inline code comments (among other problems). Since we're squash-merging PRs anyway there's usually not much reason to force-push, and it makes reviewers' lives easier if you don't. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A few questions, but no more blocking comments from me. Thanks @AndrewAXue!
ext/opentelemetry-exporter-cloud-trace/src/opentelemetry/exporter/cloud_trace/__init__.py
Show resolved
Hide resolved
ext/opentelemetry-exporter-cloud-trace/src/opentelemetry/exporter/cloud_trace/__init__.py
Outdated
Show resolved
Hide resolved
ext/opentelemetry-exporter-cloud-trace/src/opentelemetry/exporter/cloud_trace/__init__.py
Outdated
Show resolved
Hide resolved
9aa997f
to
7506644
Compare
@toumorokoshi if this LGTY it should be ready to merge. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Great, thanks!
ext/opentelemetry-exporter-cloud-trace/src/opentelemetry/exporter/cloud_trace/__init__.py
Outdated
Show resolved
Hide resolved
dropped_links = len(links) - MAX_NUM_LINKS | ||
links = links[:MAX_NUM_LINKS] | ||
for link in links: | ||
if len(link.attributes) > MAX_LINK_ATTRS: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this chunk of code (along with the warning) could be refactored into _extract_attributes.
I see some boilerplate here and extract_events that includes the max check then logs a warning. This feels intuitive to me to put the warning around some attributes will be dropped into the _extract_attributes itself, or maybe even the BoundedDict itself.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hey Yusuke, so I also noticed that the max length check + warning is repeated in extract_events and extract_links. The reason I kept it that way is so I can print more informative warnings then just "Too many attributes, dropped some", doing it this way I can say which event or link had too many attributes.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
sounds good, although it feels like that's still achievable after a refactor.
7506644
to
a0ffad6
Compare
Co-authored-by: Cheng-Lung Sung <[email protected]>
Addresses #287.
Adding a Cloud Trace (formerly known as stackdriver) trace exporter.