Improve RetrieveChat #6

thinkall · 2023-09-22T08:26:50Z

Why are these changes needed?

Use upsert in batch of 40000 to improve chromadb upsert stability, avoid error for super large corpus.
Improve Update Context logic, support customizing trigger words customized_answer_prefix. If customized_answer_prefix is given and it is not in the answer, Update Context will also be triggered.
Add a parameter no_update_context to stop Update Context, useful for ablation experiments.
Add an example of customizing prompt, trigger word and few-shot learning to RetrieveChat notebook.
Update tests.

Related issue number

Checks

I've included any doc changes needed for https://microsoft.github.io/autogen/. See https://microsoft.github.io/autogen/docs/Contribute#documentation to build and test documentation locally.
I've added tests (if relevant) corresponding to the changes introduced in this PR.
I've made sure all auto checks have passed.

codecov-commenter · 2023-09-22T08:29:59Z

Codecov Report

Merging #6 (596ff65) into main (6f54631) will increase coverage by 0.94%.
Report is 11 commits behind head on main.
The diff coverage is 20.96%.

@@            Coverage Diff             @@
##             main       #6      +/-   ##
==========================================
+ Coverage   34.22%   35.16%   +0.94%     
==========================================
  Files          17       17              
  Lines        1911     1965      +54     
  Branches      416      465      +49     
==========================================
+ Hits          654      691      +37     
- Misses       1207     1223      +16     
- Partials       50       51       +1

Flag	Coverage Δ
unittests	`35.11% <20.96%> (+0.94%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files	Coverage Δ
autogen/code_utils.py	`45.53% <92.30%> (+2.19%)`	⬆️
autogen/retrieve_utils.py	`14.78% <0.00%> (+14.78%)`	⬆️
...gen/agentchat/contrib/retrieve_user_proxy_agent.py	`4.46% <2.17%> (+3.74%)`	⬆️

sonichi · 2023-09-22T13:42:25Z

This PR decreases coverage. Can we avoid that?

thinkall · 2023-09-22T15:00:56Z

This PR decreases coverage. Can we avoid that?

I see test/agentchat/test_retrievechat.py is skipped, is there any concerns for these tests?

sonichi · 2023-09-22T15:25:39Z

This PR decreases coverage. Can we avoid that?

I see test/agentchat/test_retrievechat.py is skipped, is there any concerns for these tests?

Some tests are skipped because chromadb is not installed in [test]. We can add it.
The test that depends on OpenAI should check whether openai is installed and skip if not, like other tests.

autogen/agentchat/contrib/retrieve_user_proxy_agent.py

thinkall · 2023-09-25T00:25:20Z

One last change we need: bump the version of autogen and the version number in the notebook. Then I'll need to make a new release. Otherwise the notebook will not work.

Updated to 0.1.2

pcdeadeasy

I saw a chunk of code that was commented out in generate_init_message.
Also consider making _generate_retrieve_user_reply more readable.

…_reply

sonichi · 2023-09-25T15:53:33Z

autogen/agentchat/contrib/retrieve_user_proxy_agent.py

@@ -230,6 +230,11 @@ def _generate_retrieve_user_reply(
        sender: Optional[Agent] = None,
        config: Optional[Any] = None,
    ) -> Tuple[bool, Union[str, Dict, None]]:
+        """In this function, we will update the context and reset the conversation based on different conditions.
+        We'll update the context and reset the conversation if no_update_context is False and either of the following:


It'll be easier to think of "update_context is True" than "no_update_context is False".

pcdeadeasy

Changes look good to me. once suggestion that you could consider is to rename the rerieve_user_proxy_agent.py as RAG_user_proxy_agent.py.

* Upsert in batch * Improve update context, support customized answer prefix * Update tests * Update intermediate answer * Fix duplicate intermediate answer, add example 6 to notebook * Add notebook results * Works better without intermediate answers in the context * Bump version to 0.1.2 * Remove commented code and add descriptions to _generate_retrieve_user_reply --------- Co-authored-by: Qingyun Wu <[email protected]>

@astroalek

* intial commit for aws-bedrock * format * converse setup for model req-response * Renamed to bedrock.py, updated parameter parsing, system message extraction, client class incorporation * Established Bedrock class based on @astroalek and @ChristianT's code, added ability to disable system prompt separation * Image parsing and removing access credential checks * Added tests, added additional parameter support * Amazon Bedrock documentation * Moved client parameters to init, align parameter names with Anthropic, spelling fix, remove unnecessary imports, use base and additional parameters, update documentation * Tidy up comments * Minor typo fix * Correct comment re aws_region --------- Co-authored-by: Hk669 <[email protected]> Co-authored-by: HRUSHIKESH DOKALA <[email protected]>

add envvars in all projects

* namespace fixes + remove skills definitios from Actors project * add waf context to actors * deploy to Azure WIP * add bicep for gh-flow and cosmos * azure deploy fixes * azure deploy WIP

* Renamed TeamOne to MagenticOne * Updated uv.lock * Fixed workflows.

Upsert in batch

d8af774

thinkall had a problem deploying to openai September 22, 2023 08:26 — with GitHub Actions Failure

thinkall requested a review from sonichi September 22, 2023 08:28

Improve update context, support customized answer prefix

8750c10

thinkall had a problem deploying to openai September 22, 2023 14:52 — with GitHub Actions Failure

Update tests

ed15da5

thinkall had a problem deploying to openai September 22, 2023 15:33 — with GitHub Actions Failure

sonichi reviewed Sep 22, 2023

View reviewed changes

autogen/agentchat/contrib/retrieve_user_proxy_agent.py Outdated Show resolved Hide resolved

autogen/agentchat/contrib/retrieve_user_proxy_agent.py Show resolved Hide resolved

thinkall added 2 commits September 23, 2023 16:44

Update intermediate answer

6bd0a0b

Fix duplicate intermediate answer, add example 6 to notebook

d1d87b9

thinkall had a problem deploying to openai September 23, 2023 09:37 — with GitHub Actions Failure

Add notebook results

e288e30

thinkall had a problem deploying to openai September 23, 2023 09:51 — with GitHub Actions Failure

Works better without intermediate answers in the context

b9c82b6

thinkall had a problem deploying to openai September 23, 2023 09:59 — with GitHub Actions Failure

thinkall had a problem deploying to openai September 25, 2023 00:25 — with GitHub Actions Failure

pcdeadeasy requested changes Sep 25, 2023

View reviewed changes

Remove commented code and add descriptions to _generate_retrieve_user…

596ff65

…_reply

thinkall temporarily deployed to openai September 25, 2023 14:32 — with GitHub Actions Inactive

thinkall requested a review from pcdeadeasy September 25, 2023 14:32

sonichi reviewed Sep 25, 2023

View reviewed changes

pcdeadeasy approved these changes Sep 26, 2023

View reviewed changes

sonichi enabled auto-merge September 26, 2023 19:59

sonichi disabled auto-merge September 26, 2023 19:59

sonichi enabled auto-merge September 26, 2023 21:17

Merge branch 'main' into improve_retrieve

199a38a

sonichi added this pull request to the merge queue Sep 27, 2023

qingyun-wu had a problem deploying to openai September 27, 2023 01:22 — with GitHub Actions Failure

Merged via the queue into main with commit 1108818 Sep 27, 2023
16 of 19 checks passed

This was referenced Sep 27, 2023

Changes look good to me. once suggestion that you could consider is to rename the rerieve_user_proxy_agent.py as **RAG**_user_proxy_agent.py. #23

Closed

It'll be easier to think of "update_context is True" than "no_update_context is False". #24

Closed

thinkall deleted the improve_retrieve branch September 27, 2023 01:57

jackgerrits pushed a commit that referenced this pull request Oct 2, 2024

Merge pull request #6 from microsoft/gh-workflow

86556f9

add envvars in all projects

jackgerrits added a commit that referenced this pull request Oct 2, 2024

fix spelling error (#6)

f80c42e

jackgerrits pushed a commit that referenced this pull request Oct 2, 2024

Renamed TeamOne to MagenticOne (#6)

50ed097

* Renamed TeamOne to MagenticOne * Updated uv.lock * Fixed workflows.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improve RetrieveChat #6

Improve RetrieveChat #6

thinkall commented Sep 22, 2023 •

edited

Loading

codecov-commenter commented Sep 22, 2023 •

edited

Loading

sonichi commented Sep 22, 2023

thinkall commented Sep 22, 2023

sonichi commented Sep 22, 2023

thinkall commented Sep 25, 2023

pcdeadeasy left a comment

sonichi Sep 25, 2023

pcdeadeasy left a comment

Improve RetrieveChat #6

Improve RetrieveChat #6

Conversation

thinkall commented Sep 22, 2023 • edited Loading

Why are these changes needed?

Related issue number

Checks

codecov-commenter commented Sep 22, 2023 • edited Loading

Codecov Report

sonichi commented Sep 22, 2023

thinkall commented Sep 22, 2023

sonichi commented Sep 22, 2023

thinkall commented Sep 25, 2023

pcdeadeasy left a comment

Choose a reason for hiding this comment

sonichi Sep 25, 2023

Choose a reason for hiding this comment

pcdeadeasy left a comment

Choose a reason for hiding this comment

thinkall commented Sep 22, 2023 •

edited

Loading

codecov-commenter commented Sep 22, 2023 •

edited

Loading