Handle hallucinated tool execution requests

[JanHuege]

Not sure why I needed to add the quarkus-junit5 dependency for my test to run.

Added behavior to filter out hallucinated toolExecutionRequests.

Had to add some reflection to write a test with mocked client in OllamaChatLanguageModel. I am happy about suggestions.

Also I am not happy with the assertions in the test "doesNotCrashIfToolNotPresent". But I did not find out how to get the content of the messagebuilder from "chatResponseWithHalucinatedTool" to be present in the resultin ChatResponse. I think it has something todo with missing ToolExecutors but added breakpoints into the ToolExecutors present in the repo did not help since they didn't hit.

Fixes #1232 (hallucinated toolExecutionRequests)

[geoand]

Thanks a lot!

I'll have a closer look tomorrow, but for now I added a comment about the test

[geoand]

As for the tests, I would much rather prefer to have a tool test like the ones in the OpenAI module

[JanHuege]

I moved the tests and I've rewritten them to match the wiremock style of the others in ollama but saw your latest remark after I was done.

Are you talking about ToolBoxTest and others in quarkus-langchain4j-openai-deployment ?

I am not sure if I understand the setup of those.

[geoand]

Are you talking about ToolBoxTest and others in quarkus-langchain4j-openai-deployment ?

Right. It's similar to the other Wiremock tests and the idea is to be able to track whether a tool has been called or not.

[JanHuege]

I have added a scenario test somewhat equal to the ones in the openai package. I had to add the called property as a static one since I was unable to use an AtomicInteger like in ToolsApplicationScopedWithInterceptorTest.java since injecting the Tool creates a second AppScoped Instance making it impossible to check if called is 1. So I helped myself with this somewhat hacky solution. If you got a nicer way of ensuring the call with something like a spy I'd appreciate it.

I kept the other ones since I feel like checking/asserting the filtering of the execution is somewhat neccessary to ensure that calls are filtered out. For me without the other testcases this would cause blindspots.

[geoand]

Thanks!

[JanHuege]

Fixed/ran the formatting

🌘 This workflow status is outdated as a new workflow run has been triggered.

Status for workflow Build (on pull request)

This is the status report for running Build (on pull request) on commit 62bf751.

Failing Jobs

Status Name Step Failures Logs Raw logs
✖ Quick Build Build with Maven Failures Logs Raw logs

Failures

⚙️ Quick Build #

- Failing: model-providers/ollama/deployment 
! Skipped: codestarts codestarts/chatbot codestarts/chatbot/deployment and 65 more

📦 model-providers/ollama/deployment

✖ Failed to execute goal net.revelc.code.formatter:formatter-maven-plugin:2.24.1:validate (default) on project quarkus-langchain4j-ollama-deployment: File '/home/runner/work/quarkus-langchain4j/quarkus-langchain4j/model-providers/ollama/deployment/src/test/java/io/quarkiverse/langchain4j/ollama/deployment/OllamaChatLanguageModelToolTest.java' has not been previously formatted. Please format file (for example by invoking `mvn -f model-providers/ollama/deployment net.revelc.code.formatter:formatter-maven-plugin:2.24.1:format`) and commit before running validation!

[JanHuege]

fixed sorting of imports

[quarkus-bot]

Status for workflow Build (on pull request)

This is the status report for running Build (on pull request) on commit 2cbebc1.

✅ The latest workflow run for the pull request has completed successfully.

It should be safe to merge provided you have a look at the other checks in the summary.