Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue regarding active learning and questions with prompts(Custom question and answering [CQA]) #3878

Open
charith-phamax opened this issue Apr 18, 2023 · 1 comment
Labels
Bot Services Required for internal Azure reporting. Do not delete. Do not change color. bug Indicates an unexpected problem or an unintended behavior. customer-reported Issue is created by anyone that is not a collaborator in the repository. MINDTREE

Comments

@charith-phamax
Copy link

Github issues for C# /JS / Java/ Python should be used for bugs and feature requests. Use Stack Overflow for general "how-to" questions.

Sample information

  1. Sample type: 48.customQABot-all-features
  2. Sample language: nodejs
  3. Sample name: Custom Q&A Bot with all features

Describe the bug

When using active learning, if a question is asked which is unclear, we get options "Did you mean? [question1][question2]" etc. If the selected question contains prompts, then on selecting anyone of those prompts, we are encountering an issue where the bot is not correctly identifying user intents and providing relevant responses. If the question selected from active learning card("did you mean" card), is asked directly and not from the card, then response is relevant.

To Reproduce

Steps to reproduce the behavior:

  1. Clone the 48.customQABot-all-features sample
  2. Keep active learning feature as TRUE for your "custom question and answering" project
  3. Test the bot by asking questions which are unclear.
  4. On getting "Did you mean? [question1][question2]" choose question which has prompts
  5. Then select one of the prompt and evaluate the result.
  6. To get correct response, directly ask the question without using the active learning card. then select the same prompt and evaluate the response.

Expected behavior

We expect the custom Q&A bot to correctly identify user questions and provide relevant responses based on the active learning and prompt features.

Screenshots

  1. This screenshot contains active learning card.
    image
  2. This screenshot contains the prompts for the selected question.
    image
  3. This screenshot contains response to the selected prompt.(not relevant response)
    image
  4. This screenshot contains the question (asked directly).
    image
  5. This screenshot contains the response to the prompt selected.
    image

Additional context

No additional context

@charith-phamax charith-phamax added bug Indicates an unexpected problem or an unintended behavior. needs-triage The issue has just been created and it has not been reviewed by the team. labels Apr 18, 2023
@ramfattah ramfattah added Bot Services Required for internal Azure reporting. Do not delete. Do not change color. customer-reported Issue is created by anyone that is not a collaborator in the repository. and removed needs-triage The issue has just been created and it has not been reviewed by the team. labels Apr 18, 2023
@Amrit10737726 Amrit10737726 self-assigned this Apr 16, 2024
@Amrit10737726
Copy link
Collaborator

It seems there's uncertainty about the relevance of tsv and how it triggers certain actions within the bot. plan is to reach out to the customer for clarification on their tsv and aims to revisit the original issue once more information is available. The complexity of the problem is acknowledged, but efforts are being made to address it effectively.

@Amrit10737726 Amrit10737726 removed their assignment Apr 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bot Services Required for internal Azure reporting. Do not delete. Do not change color. bug Indicates an unexpected problem or an unintended behavior. customer-reported Issue is created by anyone that is not a collaborator in the repository. MINDTREE
Projects
None yet
Development

No branches or pull requests

4 participants