feat(openai): Make tiktoken encoding name configurable + tiktoken usage opt-in #3289
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #3289 +/- ##
==========================================
- Coverage 79.40% 79.39% -0.02%
==========================================
Files 132 132
Lines 14281 14276 -5
Branches 2999 3002 +3
==========================================
- Hits 11340 11334 -6
+ Misses 2094 2093 -1
- Partials 847 849 +2
|
sentrivana
approved these changes
Jul 15, 2024
antonpirker
approved these changes
Jul 15, 2024
arjennienhuis
pushed a commit
to arjennienhuis/sentry-python
that referenced
this pull request
Sep 30, 2024
…ge opt-in (getsentry#3289) Make tiktoken encoding name configurable + tiktoken usage opt-in --------- Co-authored-by: Ivana Kellyer <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Tiktoken encodings sometimes make network requests, or do other stateful things. We can't guarantee what happens when you call tiktoken, so let's make it explicitly opt-in to users.
General Notes
Thank you for contributing to
sentry-python!Please add tests to validate your changes, and lint your code using
tox -e linters.Running the test suite on your PR might require maintainer approval. Some tests (AWS Lambda) additionally require a maintainer to add a special label to run and will fail if the label is not present.
For maintainers
Sensitive test suites require maintainer review to ensure that tests do not compromise our secrets. This review must be repeated after any code revisions.
Before running sensitive test suites, please carefully check the PR. Then, apply the
Trigger: tests using secretslabel. The label will be removed after any code changes to enforce our policy requiring maintainers to review all code revisions before running sensitive tests.