Skip to content

Practice 5 ‐ LLMs

Attila Ficsor edited this page Oct 9, 2024 · 1 revision

Prerequisites

Any of the following tools can be used to complete the tasks in this practice. You can choose any tool that is available to you and you are comfortable with.

Tasks

Task 1 - Get an LLM to solve Practice 2a and 2b

Aks Copilot/ChatGPT/Gemini to help you turn the code in branch practice-2a into a multi-module Gradle project.

Try to answer the followin questions:

  • Does the tool give you the same method as the one you have used in the practice?
  • How helpful is the answer with different tools?
  • What are the differences in the answers?
  • What are the similarities?
  • What are the limitations of the tools?
  • How does the answer change if you give more context to the tools?

Task 2 - Developement with LLMs

Solve the following tasks using the tools you have chosen.

  • Create EuclideanDistanceComputor interface - Similar to CosineSimilarityComputor
  • Create BaseEuclideanDistanceComputor class - Similar to BaseCosineSimilarityComputor
  • Create an abstract superclass for all similarity estimator classes (implementing the DocumentSimilarityEstimator interface)
  • Change this superclass to accept EuclideanDistanceComputor and CosineSimilarityComputor as well
  • Introduce a non-trivial error to the code and fix it using Copilot
  • Generate tests for the classes you have created

Try to answer the following questions:

  • How helpful is the tool during development?
  • Are the generated code segments usable?
  • Can we trust the generated code?
  • How useful are the generated tests?
  • Can we trust the generated tests?

Task 3 - Other languages (Refinery, Langium, etc.)

Try to solve the tasks in Practice 3 and Practice 4 using the tools you have chosen. See how the answers differ from the ones you have received in the previous practices and what are the limitations of the tools.

  • Does the chosen tool know about Refinery?
  • Can you give context to the tool about Refinery to be more helpful?
  • What are the limitations of the tools in this case?
  • Does the chosen tool know about Langium?