fix: an issue of keyword search feature in application log list

[realethanhsu]

Checklist:

Important

Please review the checklist below before submitting your pull request.

• Please open an issue before creating a PR or link to an existing issue
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I ran dev/reformat(backend) and cd web && npx lint-staged(frontend) to appease the lint gods

Description

If there are multiple records in the Message table associated with the same Conversation, and some of these records contain target keywords, it can lead to duplicate Conversation records in the final query results. This results in wrong data counts and pagination issues. The revised code employs Conversation.id.in_(message_subquery) as a filter condition to ensure that only unique Conversations with related Messages are returned. This approach prevents duplicate Conversation records from appearing in the query results, thereby ensuring the accuracy of the log list and the proper functioning of the pagination feature.

Fixes #7815

Type of Change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update, included: Dify Document
• Improvement, including but not limited to code refactoring, performance optimization, and UI/UX improvement
• Dependency upgrade

Testing Instructions

Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

• Test A
• Test B

[crazywoola]

Please fix the errors in the CI.

[realethanhsu]

OK, I will finish this work today.

[realethanhsu]

@crazywoola done.

[takatost]

@realethanhsu This PR being modified to a full table scan statement, which resulted in a slow query, it could cause the database to go down.

The specific SQL that caused the full table scan is as follows:

...
conversations.id IN 
  (
    SELECT anon_2.conversation_id 
    FROM 
      (
	    SELECT messages.conversation_id AS conversation_id 
	    FROM messages 
	    WHERE messages.app_id = 'xxxx-xxxx-xxxx-xxxx-xxxx'::UUID
	    AND messages.query ILIKE '%abc%' OR messages.answer ILIKE '%abc%'
	  ) AS anon_2
  ) 
...

Due to the significant impact, we have rolled back for now.
Could you please check out the main branch again and optimize the SQL query based on it? Thank you very much.

[realethanhsu]

@crazywoola Sure, I will fix it soon.