Skip to content

Commit

Permalink
Add bedrock feedback into preview (#1030)
Browse files Browse the repository at this point in the history 
* Add AWS Bedrock testing infrastructure

* Squashed commit of the following:

commit 2834663
Author: Timothy Pansino <[email protected]>
Date:   Mon Oct 9 17:42:05 2023 -0700

    OpenAI Mock Backend (#929)

    * Add mock external openai server

    * Add mocked OpenAI server fixtures

    * Set up recorded responses.

    * Clean mock server to depend on http server

    * Linting

    * Pin flask version for flask restx tests. (#931)

    * Ignore new redis methods. (#932)

    Co-authored-by: Lalleh Rafeei <[email protected]>

    * Remove approved paths

    * Update CI Image (#930)

    * Update available python versions in CI

    * Update makefile with overrides

    * Fix default branch detection for arm builds

    ---------

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

    * Add mocking for embedding endpoint

    * [Mega-Linter] Apply linters fixes

    * Add ratelimit headers

    * [Mega-Linter] Apply linters fixes

    * Only get package version once (#928)

    * Only get package version once

    * Add disconnect method

    * Add disconnect method

    ---------

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

    * Add datalib dependency for embedding testing.

    * Add OpenAI Test Infrastructure (#926)

    * Add openai to tox

    * Add OpenAI test files.

    * Add test functions.

    * [Mega-Linter] Apply linters fixes

    ---------

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
    Co-authored-by: mergify[bot] <mergify[bot]@users.noreply.github.com>

    * Add mock external openai server

    * Add mocked OpenAI server fixtures

    * Set up recorded responses.

    * Clean mock server to depend on http server

    * Linting

    * Remove approved paths

    * Add mocking for embedding endpoint

    * [Mega-Linter] Apply linters fixes

    * Add ratelimit headers

    * [Mega-Linter] Apply linters fixes

    * Add datalib dependency for embedding testing.

    ---------

    Co-authored-by: Uma Annamalai <[email protected]>
    Co-authored-by: Lalleh Rafeei <[email protected]>
    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
    Co-authored-by: TimPansino <[email protected]>
    Co-authored-by: Hannah Stepanek <[email protected]>
    Co-authored-by: mergify[bot] <mergify[bot]@users.noreply.github.com>

commit db63d45
Author: Uma Annamalai <[email protected]>
Date:   Mon Oct 2 15:31:38 2023 -0700

    Add OpenAI Test Infrastructure (#926)

    * Add openai to tox

    * Add OpenAI test files.

    * Add test functions.

    * [Mega-Linter] Apply linters fixes

    ---------

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
    Co-authored-by: mergify[bot] <mergify[bot]@users.noreply.github.com>

* Squashed commit of the following:

commit 182c7a8
Author: Uma Annamalai <[email protected]>
Date:   Fri Oct 13 10:12:55 2023 -0700

    Add request/ response IDs.

commit f6d13f8
Author: Uma Annamalai <[email protected]>
Date:   Thu Oct 12 13:23:39 2023 -0700

    Test cleanup.

commit d057663
Author: Uma Annamalai <[email protected]>
Date:   Tue Oct 10 10:23:00 2023 -0700

    Remove commented code.

commit dd29433
Author: Uma Annamalai <[email protected]>
Date:   Tue Oct 10 10:19:01 2023 -0700

    Add openai sync instrumentation.

commit 2834663
Author: Timothy Pansino <[email protected]>
Date:   Mon Oct 9 17:42:05 2023 -0700

    OpenAI Mock Backend (#929)

    * Add mock external openai server

    * Add mocked OpenAI server fixtures

    * Set up recorded responses.

    * Clean mock server to depend on http server

    * Linting

    * Pin flask version for flask restx tests. (#931)

    * Ignore new redis methods. (#932)

    Co-authored-by: Lalleh Rafeei <[email protected]>

    * Remove approved paths

    * Update CI Image (#930)

    * Update available python versions in CI

    * Update makefile with overrides

    * Fix default branch detection for arm builds

    ---------

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

    * Add mocking for embedding endpoint

    * [Mega-Linter] Apply linters fixes

    * Add ratelimit headers

    * [Mega-Linter] Apply linters fixes

    * Only get package version once (#928)

    * Only get package version once

    * Add disconnect method

    * Add disconnect method

    ---------

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

    * Add datalib dependency for embedding testing.

    * Add OpenAI Test Infrastructure (#926)

    * Add openai to tox

    * Add OpenAI test files.

    * Add test functions.

    * [Mega-Linter] Apply linters fixes

    ---------

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
    Co-authored-by: mergify[bot] <mergify[bot]@users.noreply.github.com>

    * Add mock external openai server

    * Add mocked OpenAI server fixtures

    * Set up recorded responses.

    * Clean mock server to depend on http server

    * Linting

    * Remove approved paths

    * Add mocking for embedding endpoint

    * [Mega-Linter] Apply linters fixes

    * Add ratelimit headers

    * [Mega-Linter] Apply linters fixes

    * Add datalib dependency for embedding testing.

    ---------

    Co-authored-by: Uma Annamalai <[email protected]>
    Co-authored-by: Lalleh Rafeei <[email protected]>
    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
    Co-authored-by: TimPansino <[email protected]>
    Co-authored-by: Hannah Stepanek <[email protected]>
    Co-authored-by: mergify[bot] <mergify[bot]@users.noreply.github.com>

commit db63d45
Author: Uma Annamalai <[email protected]>
Date:   Mon Oct 2 15:31:38 2023 -0700

    Add OpenAI Test Infrastructure (#926)

    * Add openai to tox

    * Add OpenAI test files.

    * Add test functions.

    * [Mega-Linter] Apply linters fixes

    ---------

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
    Co-authored-by: mergify[bot] <mergify[bot]@users.noreply.github.com>

* TEMP

* Bedrock titan extraction nearly complete

* Bedrock Testing Infrastructure (#937)

* Add AWS Bedrock testing infrastructure

* Cache Package Version Lookups (#946)

* Cache _get_package_version

* Add Python 2.7 support to get_package_version caching

* [Mega-Linter] Apply linters fixes

* Bump tests

---------

Co-authored-by: SlavaSkvortsov <[email protected]>
Co-authored-by: TimPansino <[email protected]>

* Fix Redis Generator Methods (#947)

* Fix scan_iter for redis

* Replace generator methods

* Update instance info instrumentation

* Remove mistake from uninstrumented methods

* Add skip condition to asyncio generator tests

* Add skip condition to asyncio generator tests

---------

Co-authored-by: Lalleh Rafeei <[email protected]>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

* Automatic RPM System Updates (#948)

* Checkout old action

* Adding RPM action

* Add dry run

* Incorporating action into workflow

* Wire secret into custom action

* Enable action

* Correct action name

* Fix syntax

* Fix quoting issues

* Drop pre-verification. Does not work on python

* Fix merge artifact

* Remove OpenAI references

---------

Co-authored-by: Uma Annamalai <[email protected]>
Co-authored-by: SlavaSkvortsov <[email protected]>
Co-authored-by: TimPansino <[email protected]>
Co-authored-by: Lalleh Rafeei <[email protected]>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

* Cleaning up titan bedrock implementation

* TEMP

* Tests for bedrock passing

Co-authored-by: Lalleh Rafeei <[email protected]>

* Cleaned up titan testing

Co-authored-by: Lalleh Rafeei <[email protected]>
Co-authored-by: Hannah Stepanek <[email protected]>

* Parametrized bedrock testing

* Add support for AI21-J2 models

* Change to dynamic no conversation id events

* Drop all openai refs

* [Mega-Linter] Apply linters fixes

* Adding response_id and response_model

* Apply suggestions from code review

* Remove unused import

* Bedrock Sync Chat Completion Instrumentation (#953)

* Add AWS Bedrock testing infrastructure

* Squashed commit of the following:

commit 2834663
Author: Timothy Pansino <[email protected]>
Date:   Mon Oct 9 17:42:05 2023 -0700

    OpenAI Mock Backend (#929)

    * Add mock external openai server

    * Add mocked OpenAI server fixtures

    * Set up recorded responses.

    * Clean mock server to depend on http server

    * Linting

    * Pin flask version for flask restx tests. (#931)

    * Ignore new redis methods. (#932)

    Co-authored-by: Lalleh Rafeei <[email protected]>

    * Remove approved paths

    * Update CI Image (#930)

    * Update available python versions in CI

    * Update makefile with overrides

    * Fix default branch detection for arm builds

    ---------

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

    * Add mocking for embedding endpoint

    * [Mega-Linter] Apply linters fixes

    * Add ratelimit headers

    * [Mega-Linter] Apply linters fixes

    * Only get package version once (#928)

    * Only get package version once

    * Add disconnect method

    * Add disconnect method

    ---------

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

    * Add datalib dependency for embedding testing.

    * Add OpenAI Test Infrastructure (#926)

    * Add openai to tox

    * Add OpenAI test files.

    * Add test functions.

    * [Mega-Linter] Apply linters fixes

    ---------

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
    Co-authored-by: mergify[bot] <mergify[bot]@users.noreply.github.com>

    * Add mock external openai server

    * Add mocked OpenAI server fixtures

    * Set up recorded responses.

    * Clean mock server to depend on http server

    * Linting

    * Remove approved paths

    * Add mocking for embedding endpoint

    * [Mega-Linter] Apply linters fixes

    * Add ratelimit headers

    * [Mega-Linter] Apply linters fixes

    * Add datalib dependency for embedding testing.

    ---------

    Co-authored-by: Uma Annamalai <[email protected]>
    Co-authored-by: Lalleh Rafeei <[email protected]>
    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
    Co-authored-by: TimPansino <[email protected]>
    Co-authored-by: Hannah Stepanek <[email protected]>
    Co-authored-by: mergify[bot] <mergify[bot]@users.noreply.github.com>

commit db63d45
Author: Uma Annamalai <[email protected]>
Date:   Mon Oct 2 15:31:38 2023 -0700

    Add OpenAI Test Infrastructure (#926)

    * Add openai to tox

    * Add OpenAI test files.

    * Add test functions.

    * [Mega-Linter] Apply linters fixes

    ---------

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
    Co-authored-by: mergify[bot] <mergify[bot]@users.noreply.github.com>

* Squashed commit of the following:

commit 182c7a8
Author: Uma Annamalai <[email protected]>
Date:   Fri Oct 13 10:12:55 2023 -0700

    Add request/ response IDs.

commit f6d13f8
Author: Uma Annamalai <[email protected]>
Date:   Thu Oct 12 13:23:39 2023 -0700

    Test cleanup.

commit d057663
Author: Uma Annamalai <[email protected]>
Date:   Tue Oct 10 10:23:00 2023 -0700

    Remove commented code.

commit dd29433
Author: Uma Annamalai <[email protected]>
Date:   Tue Oct 10 10:19:01 2023 -0700

    Add openai sync instrumentation.

commit 2834663
Author: Timothy Pansino <[email protected]>
Date:   Mon Oct 9 17:42:05 2023 -0700

    OpenAI Mock Backend (#929)

    * Add mock external openai server

    * Add mocked OpenAI server fixtures

    * Set up recorded responses.

    * Clean mock server to depend on http server

    * Linting

    * Pin flask version for flask restx tests. (#931)

    * Ignore new redis methods. (#932)

    Co-authored-by: Lalleh Rafeei <[email protected]>

    * Remove approved paths

    * Update CI Image (#930)

    * Update available python versions in CI

    * Update makefile with overrides

    * Fix default branch detection for arm builds

    ---------

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

    * Add mocking for embedding endpoint

    * [Mega-Linter] Apply linters fixes

    * Add ratelimit headers

    * [Mega-Linter] Apply linters fixes

    * Only get package version once (#928)

    * Only get package version once

    * Add disconnect method

    * Add disconnect method

    ---------

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

    * Add datalib dependency for embedding testing.

    * Add OpenAI Test Infrastructure (#926)

    * Add openai to tox

    * Add OpenAI test files.

    * Add test functions.

    * [Mega-Linter] Apply linters fixes

    ---------

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
    Co-authored-by: mergify[bot] <mergify[bot]@users.noreply.github.com>

    * Add mock external openai server

    * Add mocked OpenAI server fixtures

    * Set up recorded responses.

    * Clean mock server to depend on http server

    * Linting

    * Remove approved paths

    * Add mocking for embedding endpoint

    * [Mega-Linter] Apply linters fixes

    * Add ratelimit headers

    * [Mega-Linter] Apply linters fixes

    * Add datalib dependency for embedding testing.

    ---------

    Co-authored-by: Uma Annamalai <[email protected]>
    Co-authored-by: Lalleh Rafeei <[email protected]>
    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
    Co-authored-by: TimPansino <[email protected]>
    Co-authored-by: Hannah Stepanek <[email protected]>
    Co-authored-by: mergify[bot] <mergify[bot]@users.noreply.github.com>

commit db63d45
Author: Uma Annamalai <[email protected]>
Date:   Mon Oct 2 15:31:38 2023 -0700

    Add OpenAI Test Infrastructure (#926)

    * Add openai to tox

    * Add OpenAI test files.

    * Add test functions.

    * [Mega-Linter] Apply linters fixes

    ---------

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
    Co-authored-by: mergify[bot] <mergify[bot]@users.noreply.github.com>

* Cache Package Version Lookups (#946)

* Cache _get_package_version

* Add Python 2.7 support to get_package_version caching

* [Mega-Linter] Apply linters fixes

* Bump tests

---------

Co-authored-by: SlavaSkvortsov <[email protected]>
Co-authored-by: TimPansino <[email protected]>

* Fix Redis Generator Methods (#947)

* Fix scan_iter for redis

* Replace generator methods

* Update instance info instrumentation

* Remove mistake from uninstrumented methods

* Add skip condition to asyncio generator tests

* Add skip condition to asyncio generator tests

---------

Co-authored-by: Lalleh Rafeei <[email protected]>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

* TEMP

* Automatic RPM System Updates (#948)

* Checkout old action

* Adding RPM action

* Add dry run

* Incorporating action into workflow

* Wire secret into custom action

* Enable action

* Correct action name

* Fix syntax

* Fix quoting issues

* Drop pre-verification. Does not work on python

* Fix merge artifact

* Bedrock titan extraction nearly complete

* Cleaning up titan bedrock implementation

* TEMP

* Tests for bedrock passing

Co-authored-by: Lalleh Rafeei <[email protected]>

* Cleaned up titan testing

Co-authored-by: Lalleh Rafeei <[email protected]>
Co-authored-by: Hannah Stepanek <[email protected]>

* Parametrized bedrock testing

* Add support for AI21-J2 models

* Change to dynamic no conversation id events

* Drop all openai refs

* [Mega-Linter] Apply linters fixes

* Adding response_id and response_model

* Drop python 3.7 tests for Hypercorn (#954)

* Apply suggestions from code review

* Remove unused import

---------

Co-authored-by: Uma Annamalai <[email protected]>
Co-authored-by: SlavaSkvortsov <[email protected]>
Co-authored-by: TimPansino <[email protected]>
Co-authored-by: Lalleh Rafeei <[email protected]>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Lalleh Rafeei <[email protected]>
Co-authored-by: Hannah Stepanek <[email protected]>
Co-authored-by: Lalleh Rafeei <[email protected]>

* Initial feedback commit for botocore

* Feature bedrock cohere instrumentation (#955)

* Add AWS Bedrock testing infrastructure

* Squashed commit of the following:

commit 2834663
Author: Timothy Pansino <[email protected]>
Date:   Mon Oct 9 17:42:05 2023 -0700

    OpenAI Mock Backend (#929)

    * Add mock external openai server

    * Add mocked OpenAI server fixtures

    * Set up recorded responses.

    * Clean mock server to depend on http server

    * Linting

    * Pin flask version for flask restx tests. (#931)

    * Ignore new redis methods. (#932)

    Co-authored-by: Lalleh Rafeei <[email protected]>

    * Remove approved paths

    * Update CI Image (#930)

    * Update available python versions in CI

    * Update makefile with overrides

    * Fix default branch detection for arm builds

    ---------

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

    * Add mocking for embedding endpoint

    * [Mega-Linter] Apply linters fixes

    * Add ratelimit headers

    * [Mega-Linter] Apply linters fixes

    * Only get package version once (#928)

    * Only get package version once

    * Add disconnect method

    * Add disconnect method

    ---------

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

    * Add datalib dependency for embedding testing.

    * Add OpenAI Test Infrastructure (#926)

    * Add openai to tox

    * Add OpenAI test files.

    * Add test functions.

    * [Mega-Linter] Apply linters fixes

    ---------

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
    Co-authored-by: mergify[bot] <mergify[bot]@users.noreply.github.com>

    * Add mock external openai server

    * Add mocked OpenAI server fixtures

    * Set up recorded responses.

    * Clean mock server to depend on http server

    * Linting

    * Remove approved paths

    * Add mocking for embedding endpoint

    * [Mega-Linter] Apply linters fixes

    * Add ratelimit headers

    * [Mega-Linter] Apply linters fixes

    * Add datalib dependency for embedding testing.

    ---------

    Co-authored-by: Uma Annamalai <[email protected]>
    Co-authored-by: Lalleh Rafeei <[email protected]>
    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
    Co-authored-by: TimPansino <[email protected]>
    Co-authored-by: Hannah Stepanek <[email protected]>
    Co-authored-by: mergify[bot] <mergify[bot]@users.noreply.github.com>

commit db63d45
Author: Uma Annamalai <[email protected]>
Date:   Mon Oct 2 15:31:38 2023 -0700

    Add OpenAI Test Infrastructure (#926)

    * Add openai to tox

    * Add OpenAI test files.

    * Add test functions.

    * [Mega-Linter] Apply linters fixes

    ---------

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
    Co-authored-by: mergify[bot] <mergify[bot]@users.noreply.github.com>

* Squashed commit of the following:

commit 182c7a8
Author: Uma Annamalai <[email protected]>
Date:   Fri Oct 13 10:12:55 2023 -0700

    Add request/ response IDs.

commit f6d13f8
Author: Uma Annamalai <[email protected]>
Date:   Thu Oct 12 13:23:39 2023 -0700

    Test cleanup.

commit d057663
Author: Uma Annamalai <[email protected]>
Date:   Tue Oct 10 10:23:00 2023 -0700

    Remove commented code.

commit dd29433
Author: Uma Annamalai <[email protected]>
Date:   Tue Oct 10 10:19:01 2023 -0700

    Add openai sync instrumentation.

commit 2834663
Author: Timothy Pansino <[email protected]>
Date:   Mon Oct 9 17:42:05 2023 -0700

    OpenAI Mock Backend (#929)

    * Add mock external openai server

    * Add mocked OpenAI server fixtures

    * Set up recorded responses.

    * Clean mock server to depend on http server

    * Linting

    * Pin flask version for flask restx tests. (#931)

    * Ignore new redis methods. (#932)

    Co-authored-by: Lalleh Rafeei <[email protected]>

    * Remove approved paths

    * Update CI Image (#930)

    * Update available python versions in CI

    * Update makefile with overrides

    * Fix default branch detection for arm builds

    ---------

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

    * Add mocking for embedding endpoint

    * [Mega-Linter] Apply linters fixes

    * Add ratelimit headers

    * [Mega-Linter] Apply linters fixes

    * Only get package version once (#928)

    * Only get package version once

    * Add disconnect method

    * Add disconnect method

    ---------

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

    * Add datalib dependency for embedding testing.

    * Add OpenAI Test Infrastructure (#926)

    * Add openai to tox

    * Add OpenAI test files.

    * Add test functions.

    * [Mega-Linter] Apply linters fixes

    ---------

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
    Co-authored-by: mergify[bot] <mergify[bot]@users.noreply.github.com>

    * Add mock external openai server

    * Add mocked OpenAI server fixtures

    * Set up recorded responses.

    * Clean mock server to depend on http server

    * Linting

    * Remove approved paths

    * Add mocking for embedding endpoint

    * [Mega-Linter] Apply linters fixes

    * Add ratelimit headers

    * [Mega-Linter] Apply linters fixes

    * Add datalib dependency for embedding testing.

    ---------

    Co-authored-by: Uma Annamalai <[email protected]>
    Co-authored-by: Lalleh Rafeei <[email protected]>
    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
    Co-authored-by: TimPansino <[email protected]>
    Co-authored-by: Hannah Stepanek <[email protected]>
    Co-authored-by: mergify[bot] <mergify[bot]@users.noreply.github.com>

commit db63d45
Author: Uma Annamalai <[email protected]>
Date:   Mon Oct 2 15:31:38 2023 -0700

    Add OpenAI Test Infrastructure (#926)

    * Add openai to tox

    * Add OpenAI test files.

    * Add test functions.

    * [Mega-Linter] Apply linters fixes

    ---------

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
    Co-authored-by: mergify[bot] <mergify[bot]@users.noreply.github.com>

* TEMP

* Bedrock titan extraction nearly complete

* Cleaning up titan bedrock implementation

* TEMP

* Tests for bedrock passing

Co-authored-by: Lalleh Rafeei <[email protected]>

* Cleaned up titan testing

Co-authored-by: Lalleh Rafeei <[email protected]>
Co-authored-by: Hannah Stepanek <[email protected]>

* Parametrized bedrock testing

* Add support for AI21-J2 models

* Change to dynamic no conversation id events

* Add cohere model

* Remove openai instrumentation from this branch

* Remove OpenAI from newrelic/config.py

---------

Co-authored-by: Uma Annamalai <[email protected]>
Co-authored-by: Tim Pansino <[email protected]>
Co-authored-by: Lalleh Rafeei <[email protected]>
Co-authored-by: Hannah Stepanek <[email protected]>

* Bedrock feedback w/ testing for titan and jurassic models

* AWS Bedrock Embedding Instrumentation (#957)

* AWS Bedrock embedding instrumentation

* Correct symbol name

* Add support for bedrock claude (#960)

Co-authored-by: Timothy Pansino <[email protected]>

* Fix merge conflicts

* Combine Botocore Tests (#959)

* Initial file migration

* Enable DT on all span tests

* Add pytest skip for older botocore versions

* Fixup: app name merge conflict

---------

Co-authored-by: Hannah Stepanek <[email protected]>

* Add to and move feedback tests

* Handle 0.32.0.post1 version in tests (#963)

* Remove response_id dependency in bedrock

* Change API name

* Update moto

* Bedrock Error Tracing (#966)

* Cache Package Version Lookups (#946)

* Cache _get_package_version

* Add Python 2.7 support to get_package_version caching

* [Mega-Linter] Apply linters fixes

* Bump tests

---------

Co-authored-by: SlavaSkvortsov <[email protected]>
Co-authored-by: TimPansino <[email protected]>

* Fix Redis Generator Methods (#947)

* Fix scan_iter for redis

* Replace generator methods

* Update instance info instrumentation

* Remove mistake from uninstrumented methods

* Add skip condition to asyncio generator tests

* Add skip condition to asyncio generator tests

---------

Co-authored-by: Lalleh Rafeei <[email protected]>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

* Automatic RPM System Updates (#948)

* Checkout old action

* Adding RPM action

* Add dry run

* Incorporating action into workflow

* Wire secret into custom action

* Enable action

* Correct action name

* Fix syntax

* Fix quoting issues

* Drop pre-verification. Does not work on python

* Fix merge artifact

* Drop python 3.7 tests for Hypercorn (#954)

* Fix pyenv installation for devcontainer (#936)

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

* Remove duplicate kafka import hook (#956)

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

* Initial bedrock error tracing commit

* Handle 0.32.0.post1 version in tests (#963)

* Add status code to mock bedrock server

* Updating error response recording logic

* Work on bedrock errror tracing

* Chat completion error tracing

* Adding embedding error tracing

* Delete comment

* Update moto

---------

Co-authored-by: SlavaSkvortsov <[email protected]>
Co-authored-by: TimPansino <[email protected]>
Co-authored-by: Lalleh Rafeei <[email protected]>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Lalleh Rafeei <[email protected]>
Co-authored-by: Hannah Stepanek <[email protected]>

* Change ids to match other tests

* move message_ids declaration outside for loop

* Add comment to tox.ini

* Drop py27 from memcache testing.

* Drop pypy27 from memcache testing.

* Update flaskrestx testing #1004

* Remove tastypie 0.14.3 testing

* Remove tastypie 0.14.3 testing

* Remove python 3.12 support (for now)

* Remove untouched files from diff list

---------

Co-authored-by: Uma Annamalai <[email protected]>
Co-authored-by: Tim Pansino <[email protected]>
Co-authored-by: Timothy Pansino <[email protected]>
Co-authored-by: SlavaSkvortsov <[email protected]>
Co-authored-by: TimPansino <[email protected]>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Lalleh Rafeei <[email protected]>
Co-authored-by: Hannah Stepanek <[email protected]>
Co-authored-by: Hannah Stepanek <[email protected]>
  • Loading branch information
@lrafeei @umaannamalai
@TimPansino @SlavaSkvortsov @mergify @hmstepanek
10 people authored Jan 10, 2024
1 parent 7051455 commit 7f062c4
Show file tree
Hide file tree
Showing 6 changed files with 129 additions and 19 deletions.
9 changes: 6 additions & 3 deletions newrelic/api/ml_model.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -40,12 +40,15 @@ def wrap_mlmodel(model, name=None, version=None, feature_names=None, label_names

def get_llm_message_ids(response_id=None):
transaction = current_transaction()
if response_id and transaction:
if transaction:
nr_message_ids = getattr(transaction, "_nr_message_ids", {})
message_id_info = nr_message_ids.pop(response_id, ())
message_id_info = (
nr_message_ids.pop("bedrock_key", ()) if not response_id else nr_message_ids.pop(response_id, ())
)

if not message_id_info:
warnings.warn("No message ids found for %s" % response_id)
response_id_warning = "." if not response_id else " for %s." % response_id
warnings.warn("No message ids found%s" % response_id_warning)
return []

conversation_id, request_id, ids = message_id_info
Expand Down
9 changes: 8 additions & 1 deletion newrelic/hooks/external_botocore.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -97,6 +97,7 @@ def create_chat_completion_message_event(
if not transaction:
return

message_ids = []
for index, message in enumerate(input_message_list):
if response_id:
id_ = "%s-%d" % (response_id, index) # Response ID was set, append message index to it.
Expand Down Expand Up @@ -128,6 +129,7 @@ def create_chat_completion_message_event(
id_ = "%s-%d" % (response_id, index) # Response ID was set, append message index to it.
else:
id_ = str(uuid.uuid4()) # No response IDs, use random UUID
message_ids.append(id_)

chat_completion_message_dict = {
"id": id_,
Expand All @@ -147,6 +149,7 @@ def create_chat_completion_message_event(
"is_response": True,
}
transaction.record_custom_event("LlmChatCompletionMessage", chat_completion_message_dict)
return (conversation_id, request_id, message_ids)


def extract_bedrock_titan_text_model(request_body, response_body=None):
Expand Down Expand Up @@ -577,7 +580,7 @@ def handle_chat_completion_event(

transaction.record_custom_event("LlmChatCompletionSummary", chat_completion_summary_dict)

create_chat_completion_message_event(
message_ids = create_chat_completion_message_event(
transaction=transaction,
app_name=settings.app_name,
input_message_list=input_message_list,
Expand All @@ -591,6 +594,10 @@ def handle_chat_completion_event(
response_id=response_id,
)

if not hasattr(transaction, "_nr_message_ids"):
transaction._nr_message_ids = {}
transaction._nr_message_ids["bedrock_key"] = message_ids


CUSTOM_TRACE_POINTS = {
("sns", "publish"): message_trace("SNS", "Produce", "Topic", extract(("TopicArn", "TargetArn"), "PhoneNumber")),
Expand Down
87 changes: 87 additions & 0 deletions tests/external_botocore/_test_bedrock_chat_completion.py
View file
Original file line number Diff line number Diff line change
@@ -1,3 +1,17 @@
# Copyright 2010 New Relic, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

chat_completion_payload_templates = {
"amazon.titan-text-express-v1": '{ "inputText": "%s", "textGenerationConfig": {"temperature": %f, "maxTokenCount": %d }}',
"ai21.j2-mid-v1": '{"prompt": "%s", "temperature": %f, "maxTokens": %d}',
Expand All @@ -6,6 +20,79 @@
"meta.llama2-13b-chat-v1": '{"prompt": "%s", "temperature": %f, "max_gen_len": %d}',
}

chat_completion_get_llm_message_ids = {
"amazon.titan-text-express-v1": {
"bedrock_key": [
{
"conversation_id": "my-awesome-id",
"request_id": "03524118-8d77-430f-9e08-63b5c03a40cf",
"message_id": None, # UUID that varies with each run
},
{
"conversation_id": "my-awesome-id",
"request_id": "03524118-8d77-430f-9e08-63b5c03a40cf",
"message_id": None, # UUID that varies with each run
},
]
},
"ai21.j2-mid-v1": {
"bedrock_key": [
{
"conversation_id": "my-awesome-id",
"request_id": "c863d9fc-888b-421c-a175-ac5256baec62",
"message_id": "1234-0",
},
{
"conversation_id": "my-awesome-id",
"request_id": "c863d9fc-888b-421c-a175-ac5256baec62",
"message_id": "1234-1",
},
]
},
"anthropic.claude-instant-v1": {
"bedrock_key": [
{
"conversation_id": "my-awesome-id",
"request_id": "7b0b37c6-85fb-4664-8f5b-361ca7b1aa18",
"message_id": None, # UUID that varies with each run
},
{
"conversation_id": "my-awesome-id",
"request_id": "7b0b37c6-85fb-4664-8f5b-361ca7b1aa18",
"message_id": None, # UUID that varies with each run
},
]
},
"cohere.command-text-v14": {
"bedrock_key": [
{
"conversation_id": "my-awesome-id",
"request_id": "e77422c8-fbbf-4e17-afeb-c758425c9f97",
"message_id": "e77422c8-fbbf-4e17-afeb-c758425c9f97-0",
},
{
"conversation_id": "my-awesome-id",
"request_id": "e77422c8-fbbf-4e17-afeb-c758425c9f97",
"message_id": "e77422c8-fbbf-4e17-afeb-c758425c9f97-1",
},
]
},
"meta.llama2-13b-chat-v1": {
"bedrock_key": [
{
"conversation_id": "my-awesome-id",
"request_id": "9a64cdb0-3e82-41c7-873a-c12a77e0143a",
"message_id": "9a64cdb0-3e82-41c7-873a-c12a77e0143a-0",
},
{
"conversation_id": "my-awesome-id",
"request_id": "9a64cdb0-3e82-41c7-873a-c12a77e0143a",
"message_id": "9a64cdb0-3e82-41c7-873a-c12a77e0143a-1",
},
]
},
}

chat_completion_expected_events = {
"amazon.titan-text-express-v1": [
(
Expand Down
18 changes: 16 additions & 2 deletions tests/external_botocore/_test_bedrock_embeddings.py
View file
Original file line number Diff line number Diff line change
@@ -1,3 +1,17 @@
# Copyright 2010 New Relic, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

embedding_payload_templates = {
"amazon.titan-embed-text-v1": '{ "inputText": "%s" }',
"amazon.titan-embed-g1-text-02": '{ "inputText": "%s" }',
Expand Down Expand Up @@ -68,7 +82,7 @@
"request_id": "",
"vendor": "bedrock",
"ingest_source": "Python",
"error": True
"error": True,
},
),
],
Expand All @@ -89,7 +103,7 @@
"request_id": "",
"vendor": "bedrock",
"ingest_source": "Python",
"error": True
"error": True,
},
),
],
Expand Down
2 changes: 1 addition & 1 deletion tests/external_botocore/test_bedrock_embeddings.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,8 @@
import pytest
from _test_bedrock_embeddings import (
embedding_expected_client_errors,
embedding_expected_events,
embedding_expected_error_events,
embedding_expected_events,
embedding_payload_templates,
)
from conftest import BOTOCORE_VERSION
Expand Down
23 changes: 11 additions & 12 deletions tox.ini
View file
Original file line number Diff line number Diff line change
Expand Up @@ -67,19 +67,19 @@ envlist =
python-mlmodel_sklearn-{py37}-scikitlearn0101,
python-component_djangorestframework-py27-djangorestframework0300,
python-component_djangorestframework-{py37,py38,py39,py310,py311}-djangorestframeworklatest,
python-component_flask_rest-{py37,py38,py39,pypy38}-flaskrestxlatest,
python-component_flask_rest-py37-flaskrestx110,
python-component_flask_rest-{py38,py39,py310,py311,pypy38}-flaskrestxlatest,
python-component_flask_rest-{py27,pypy27}-flaskrestx051,
python-component_graphqlserver-{py37,py38,py39,py310,py311},
python-component_tastypie-{py27,pypy27}-tastypie0143,
python-component_tastypie-{py37,py38,py39,pypy38}-tastypie{0143,latest},
python-component_tastypie-{py37,py38,py39,pypy38}-tastypielatest,
python-coroutines_asyncio-{py37,py38,py39,py310,py311,pypy38},
python-cross_agent-{py27,py37,py38,py39,py310,py311}-{with,without}_extensions,
python-cross_agent-pypy27-without_extensions,
postgres-datastore_asyncpg-{py37,py38,py39,py310,py311},
memcached-datastore_bmemcached-{pypy27,py27,py37,py38,py39,py310,py311}-memcached030,
elasticsearchserver07-datastore_elasticsearch-{py27,py37,py38,py39,py310,py311,pypy27,pypy38}-elasticsearch07,
elasticsearchserver08-datastore_elasticsearch-{py37,py38,py39,py310,py311,pypy38}-elasticsearch08,
memcached-datastore_memcache-{py27,py37,py38,py39,py310,py311,pypy27,pypy38}-memcached01,
memcached-datastore_memcache-{py37,py38,py39,py310,py311,pypy38}-memcached01,
mysql-datastore_mysql-mysql080023-py27,
mysql-datastore_mysql-mysqllatest-{py37,py38,py39,py310,py311},
firestore-datastore_firestore-{py37,py38,py39,py310,py311},
Expand Down Expand Up @@ -177,7 +177,8 @@ deps =
adapter_gunicorn-aiohttp3: aiohttp<4.0
adapter_gunicorn-gunicorn19: gunicorn<20
adapter_gunicorn-gunicornlatest: gunicorn
adapter_hypercorn-hypercornlatest: hypercorn
; Temporarily pinned. Needs to be addressed
adapter_hypercorn-hypercornlatest: hypercorn<0.16
adapter_hypercorn-hypercorn0013: hypercorn<0.14
adapter_hypercorn-hypercorn0012: hypercorn<0.13
adapter_hypercorn-hypercorn0011: hypercorn<0.12
Expand All @@ -204,23 +205,21 @@ deps =
component_djangorestframework-djangorestframework0300: djangorestframework<3.1
component_djangorestframework-djangorestframeworklatest: Django
component_djangorestframework-djangorestframeworklatest: djangorestframework
component_flask_rest: flask
component_flask_rest: flask-restful
component_flask_rest: jinja2
component_flask_rest: itsdangerous
component_flask_rest-flaskrestxlatest: flask
component_flask_rest-flaskrestxlatest: flask-restx
; Pin Flask version until flask-restx is updated to support v3
component_flask_rest-flaskrestxlatest: flask<3.0
; flask-restx only supports Flask v3 after flask-restx v1.3.0
component_flask_rest-flaskrestx110: Flask<3.0
component_flask_rest-flaskrestx110: flask-restx<1.2
component_flask_rest-flaskrestx051: Flask<3.0
component_flask_rest-flaskrestx051: flask-restx<1.0
component_graphqlserver: graphql-server[sanic,flask]==3.0.0b5
component_graphqlserver: sanic>20
component_graphqlserver: Flask
component_graphqlserver: markupsafe<2.1
component_graphqlserver: jinja2<3.1
component_tastypie-tastypie0143: django-tastypie<0.14.4
component_tastypie-{py27,pypy27}-tastypie0143: django<1.12
component_tastypie-{py37,py38,py39,py310,py311,pypy38}-tastypie0143: django<3.0.1
component_tastypie-{py37,py38,py39,py310,py311,pypy38}-tastypie0143: asgiref<3.7.1 # asgiref==3.7.1 only suppport Python 3.10+
component_tastypie-tastypielatest: django-tastypie
component_tastypie-tastypielatest: django<4.1
component_tastypie-tastypielatest: asgiref<3.7.1 # asgiref==3.7.1 only suppport Python 3.10+
Expand Down

0 comments on commit 7f062c4

Please sign in to comment.