Generative AI support for Spring Petclinic Microservices #281

odedia · 2024-10-09T15:28:34Z

This PR adds a new microservice to support a generative ai chatbot based on Spring AI.
It supports natural language for performing activities like listing vets, listing owners, adding owners and adding pets to owners.
If the microservice is not booted, the chatbot simply response that chat is not available.
Based on https://github.com/spring-projects/spring-petclinic/tree/spring-ai
The code was adapted to keep domain boundaries - the functions now call the relevant microservice to perform the various activities as needed.
It also include an example of using RAG in the listVets function.
Thank you.

…ts listing vets, listing owners, adding owners and adding pets to owners

arey · 2024-10-09T17:55:20Z

Hi @odedia, thanks for this contribution I have to review.

Since my last email to you and @showpune, I've reworked a bit your spring-ai branch and published it in a brand new repository: https://github.com/spring-petclinic/spring-petclinic-springai
The chatbot is always active. I will push other improvements trough Pull Request.

Over the next few months I plan to make a version with Lanchain4j.

odedia · 2024-10-09T18:27:44Z

Thanks!
The purpose of this exercise is to show spring ai naturally integrating with a Microservices architecture.
the chatbot will not do anything if the genai microservice is not running.

sonarcloud · 2024-10-11T09:44:15Z

Quality Gate failed

Failed conditions
6 Security Hotspots
3.1% Duplication on New Code (required ≤ 3%)
B Reliability Rating on New Code (required ≥ A)

See analysis details on SonarCloud

Catch issues before they fail your Quality Gate with our IDE extension SonarLint

arey

Thank you @odedia for your contribution to modernizing the architecture by adding a very hype GenAI service in 2024.
I've given you a lot of feedback. Topics are open for discussion.

In order to be able to merge this PR on the main branch, to remain open to as many developers as possible, I think the sample application needs to start without an OpenAI or Azure account. The genai servide should be optional or start in LLM mode.

arey · 2024-10-11T12:51:47Z

spring-petclinic-genai-service/src/main/resources/application.yml

@@ -0,0 +1,43 @@
+spring:


suggestion: most of the configuration properties are located to the spring-petclinic-microservices-config repository, especially the docker profile. I suggest to create another PR to report it.
https://github.com/spring-petclinic/spring-petclinic-microservices-config/

arey · 2024-10-11T12:58:50Z

spring-petclinic-genai-service/src/main/resources/application.yml

+    active: production
+  config:
+    import: optional:configserver:${CONFIG_SERVER_URL:http://localhost:8888/},optional:classpath:/creds.yaml
+  #These apply when using spring-ai-azure-openai-spring-boot-starter


question: how could be the credentials managed in a Docker image exported to a registry? We have to answer this question whether we leave the config here or centralize it on the server config.

I'm not sure if we can reliably export the credentials in the config server. Since this is private sensitive data, and the config server is publicly available on Github, the creds.yaml should remain in the source code of the genai service. It is listed for ignore in .gitignore.

The config repository could be cloned in a private repository. It's URL could be provided with the CONFIG_SERVER_URL environment variable. I agree: it's not very practical.

I'm not very fan of adding the creds.yaml in a Docker image. We could use a Docker volume. But I'll prefer to use an environment variable like AZURE_OPENAI_KEY and AZURE_OPENAI_ENDPOINT. See spring-petclinic/spring-petclinic-ai@fbaf8dc#diff-54eeffbae371fcd1398d4ca5e89a1b8118208b7bb2f8ddf55c1aa2f7d98ab136R36
Developers can pass those parameteres from their IDE or from the Docker command line. What do you think about?

Yep, that would work.

arey · 2024-10-11T13:06:13Z

spring-petclinic-genai-service/src/main/resources/application.yml

+    chat:
+      client:
+        enabled: true
+    azure:


suggestion : in the spring-petclinic-ai, I used the defaultFunctions method of the ChatClient. Like this we don't depend of Azure OpenAI or OpenAPI specificities. Are you agree with this approach?
See spring-petclinic/spring-petclinic-ai#2

I have no preference one way or the other, it can be in the code instead.

I prefer this because it avoids having duplicate configuration lines/

spring.ai.azure.openai.chat.options.functions=xxx,yyy spring.ai.openai.chat.options.functions=xxx,yyy

arey · 2024-10-11T14:50:11Z

...enai-service/src/main/java/org/springframework/samples/petclinic/genai/dto/VisitDetails.java

+ */
+@Data
+@NoArgsConstructor
+public class VisitDetails {


suggestion: I propose to user records. Is there any technical constraint?
Same question for PetDetails and OwnerDetails

Records would be great, I simply opted to use the same classes that already exist in other microservices for clarity and cosistency.

Last year, we began to remove Lombok: #251
Could you please try to use record? If it's too complicated, it doesn't matter: we'll postpone the change.

arey · 2024-10-12T19:36:38Z

...src/main/java/org/springframework/samples/petclinic/api/boundary/web/FallbackController.java

+@RestController
+public class FallbackController {
+
+    @RequestMapping("/fallback")


suggestion: add POST (and GET ?) in the method attributes

arey · 2024-10-12T19:39:40Z

README.md

@@ -93,7 +93,25 @@ Each service has its own specific role and communicates via REST APIs.

 ![Spring Petclinic Microservices architecture](docs/microservices-architecture-diagram.jpg)

+## Integrating the Spring AI Chatbot


suggestion: we need to offer a solution to developers who don't have an Azure or OpenAI account and don't want to pay for it: either enable fallback by default, or disable chat, or have a local solution with ollama. What do you think @odedia ?

I began with using Llama 3.1, but the chat client became incredibly verbose and not as useful as OpenAI, that's why I switched to it.

arey · 2024-10-12T19:44:17Z

spring-petclinic-api-gateway/src/main/resources/application.yml

+          args:
+            retries: 1
+            statuses: SERVICE_UNAVAILABLE
+            methods: GET, POST


The POST looks like to be enough. Isn't it?

Seems like it, yes.

arey · 2024-10-12T19:45:44Z

spring-petclinic-api-gateway/src/main/resources/static/index.html

+
+
+<script th:src="@{/webjars/bootstrap/5.3.3/dist/js/bootstrap.bundle.min.js}"></script>
+<script src="https://cdn.jsdelivr.net/npm/marked/marked.min.js"></script>


suggestion: to be consistent with bootstrap we could declare marked as a webjar

Can you share an example for that?

Sure: spring-petclinic/spring-petclinic-ai@5eceb63

arey · 2024-10-12T19:51:08Z

There are also somme issues detected by the Sonar Quality Gate.

odedia · 2024-10-13T07:13:05Z

Sonarqube is mostly complaining about Code duplication, because I copied the data classes from existing microservices to the new one.
However this is the correct approach in a microservices architecture, you don't want to have dependencies on other microservices.
A better approach would be to use contract-based DTO generation but that's a rather large undertaking for the entire project.

odedia · 2024-10-13T07:15:39Z

I don't see a develop branch in the upstream project, is that exposed to all?

arey · 2024-10-14T06:39:34Z

Sonarqube is mostly complaining about Code duplication, because I copied the data classes from existing microservices to the new one. However this is the correct approach in a microservices architecture, you don't want to have dependencies on other microservices. A better approach would be to use contract-based DTO generation but that's a rather large undertaking for the entire project.

Putting code duplication to one side.
We can focus on the 12 issues. I can help you if you want.

arey · 2024-10-14T06:40:20Z

I don't see a develop branch in the upstream project, is that exposed to all?

I got confused. I meant the main branch

odedia · 2024-10-14T06:44:00Z

Any help would be appreciated, I will try to implement the other changes towards the weekend.

odedia added 2 commits October 9, 2024 17:08

updated git ignore

254e65b

New microservice for generative ai chatbot based on Spring AI. Suppor…

66458ad

…ts listing vets, listing owners, adding owners and adding pets to owners

arey added the enhancement label Oct 9, 2024

Update README.md

2a5d6e3

colljos mentioned this pull request Oct 11, 2024

Genai Chatbot startup failure #282

Open

arey reviewed Oct 12, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Generative AI support for Spring Petclinic Microservices #281

Generative AI support for Spring Petclinic Microservices #281

odedia commented Oct 9, 2024

arey commented Oct 9, 2024

odedia commented Oct 9, 2024

sonarcloud bot commented Oct 11, 2024

arey left a comment •

edited

Loading

arey Oct 11, 2024

arey Oct 11, 2024

odedia Oct 13, 2024

arey Oct 13, 2024

odedia Oct 13, 2024

arey Oct 11, 2024

odedia Oct 13, 2024

arey Oct 13, 2024

arey Oct 11, 2024

odedia Oct 13, 2024

arey Oct 13, 2024

arey Oct 12, 2024

odedia Oct 13, 2024

arey Oct 12, 2024

odedia Oct 13, 2024

arey Oct 12, 2024

odedia Oct 13, 2024

arey Oct 12, 2024

odedia Oct 13, 2024

arey Oct 13, 2024

arey commented Oct 12, 2024

odedia commented Oct 13, 2024

odedia commented Oct 13, 2024

arey commented Oct 14, 2024

arey commented Oct 14, 2024

odedia commented Oct 14, 2024

		@@ -93,7 +93,25 @@ Each service has its own specific role and communicates via REST APIs.

		![Spring Petclinic Microservices architecture](docs/microservices-architecture-diagram.jpg)

		## Integrating the Spring AI Chatbot



		<script th:src="@{/webjars/bootstrap/5.3.3/dist/js/bootstrap.bundle.min.js}"></script>
		<script src="https://cdn.jsdelivr.net/npm/marked/marked.min.js"></script>

Generative AI support for Spring Petclinic Microservices #281

Are you sure you want to change the base?

Generative AI support for Spring Petclinic Microservices #281

Conversation

odedia commented Oct 9, 2024

arey commented Oct 9, 2024

odedia commented Oct 9, 2024

sonarcloud bot commented Oct 11, 2024

Quality Gate failed

arey left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

arey commented Oct 12, 2024

odedia commented Oct 13, 2024

odedia commented Oct 13, 2024

arey commented Oct 14, 2024

arey commented Oct 14, 2024

odedia commented Oct 14, 2024

arey left a comment •

edited

Loading