UnitTestBot consists of three processes:
IDE process
— the process where the plugin part executes. We also call it the plugin process or the IntelliJ IDEA process.Engine process
— the process where the test generation engine executes.Instrumented process
— the process where concrete execution takes place.
These processes are built on top of the Reactive distributed communication framework (Rd) developed by JetBrains. Rd plays a crucial role in UnitTestBot machinery, so we briefly describe this library here.
To gain an insight into Rd, one should grasp these Rd concepts:
- Lifetime
- Rd entities
Rdgen
Imagine an object holding resources that should be released upon the object's death. In Java,
Closeable
and AutoCloseable
interfaces are introduced to help release resources that the object is holding.
Support for try
-with-resources in Java and Closeable.use
in Kotlin are also implemented to assure that the
resources are closed after the execution of the given block.
Though, releasing resources upon the object's death is still problematic:
- An object's lifetime can be more complicated than the scope of
Closeable.use
. - If
Closeable
is a function parameter, should we close it? - Multithreading and concurrency may lead to more complex situations.
- Considering all these issues, how should we correctly close the objects that depend on some other object's lifetime? How can we perform this task in a fast and easy way?
So, Rd introduces the concept of Lifetime
.
Note: the described relationships and behavior refer to the JVM-related part of Rd.
Lifetime
is an abstract class, with LifetimeDefinition
as its inheritor. LifetimeDefinition
has only one difference from its parent: LifetimeDefinition
can be terminated.
Each Lifetime
variable is an instance of LifetimeDefinition
(we later call it "Lifetime
instance"). You
can register callbacks in this Lifetime
instance — all of them will be executed upon the termination.
Though all Lifetime
objects are instances of LifetimeDefinition
, there are conventions for using them:
- Do not cast
Lifetime
toLifetimeDefinion
unless you are the one who createdLifetimeDefinition
. - If you introduce
LifetimeDefinition
somewhere, you should attach it to anotherLifetime
or provide the code that terminates it.
A Lifetime
instance has these useful methods:
onTermination
executes lambda/closeable when theLifetime
instance is terminated. If an instance has been already terminated, it executes lambda/closeable instantly. Termination proceeds on a thread that has invokedLifetimeDefinition.terminate
. Callbacks are executed in the reversed order, which is LIFO: the last added callback is executed first.onTerminationIfAlive
is the same asonTermination
, but the callback is executed only if theLifetime
instance isAlive
.executeIfAlive
executes lambda if theLifetime
instance isAlive
. This method guarantees that theLifetime
instance is alive (i.e. will not be terminated) during the whole time of lambda execution.createdNested
creates the childLifetimeDefinition
instance: it can be terminated if the parent instance is terminated as well; or it can be terminated separately, while the parent instance stays alive.usingNested
is the same as thecreateNested
method but behaves like theCloseable.use
pattern.
See also:
Lifetime.Eternal
is a globalLifetime
instance that is never terminated.Lifetime.Terminated
is a globalLifetime
instance that has been already terminated.status
— find more details in the LifetimeStatus.kt class from the Rd repository. There are three convenient methods:IsAlive
,IsNotAlive
,IsTerminated
.
LifetimeDefinition
instances have the terminate
method that terminates a Lifetime
instance and invokes all
the registered callbacks. If multiple concurrent terminations occur, the method may sometimes return before
executing all the callbacks because some other thread executes them.
Rd is a lightweight reactive one-to-one RPC protocol, which is cross-language as well as cross-platform. It can work on the same or different machines via the Internet.
These are some Rd entities:
Protocol
encapsulates the logic of all Rd communications. All the entities should be bound toProtocol
before being used.Protocol
containsIScheduler
, which executes a runnable instance on a different thread.RdSignal
is an entity allowing one to fire and forget. You can add a callback for every received message via theadvise(lifetime, callback)
method. There are two interfaces:ISink
that only allows advising for messages andISignal
that can alsofire
events. There is also aSignal
class with the same behavior but without remote communication.
Important: if you advise
and fire
from the same process, your callback receives not only
messages from the other process, but also the ones you fire
.
RdProperty
is a stateful property. You can get the current value and advise the callback — an advised callback is executed on a current value and every change.RdCall
is the remote procedure call.
There are RdSet
, RdMap
, and other entities.
An async
property allows you to fire
entities from any thread. Otherwise, you would need to do it from
the Protocol.scheduler
thread: all Rd entities should be bound to the Protocol
from the scheduler
thread, or you
would get an exception.
Rdgen
generates custom classes and requests that can be bound to protocol and advised. There is a special model DSL
for it.
Examples:
- Korifey — a simple one.
- Rider Unity plugin — a complicated one.
First, you need to define a Root
object: only one instance of each Root
can be assigned to Protocol
.
There is a Root
extension — Ext(YourRoot)
— where you can define custom types and model entities. You can assign
multiple Root
extensions to the Protocol
. To generate the auxiliary structures, define them as direct fields.
DSL:
structdef
is a structure with fields that cannot be bound toProtocol
but can be serialized. This structure can beopenstruct
, i.e. open for inheritance, andbasestruct
, i.e. abstract. Onlyfield
can be a member.classdef
is a class that can be bound to a model. It can haveproperty
,signal
,call
, etc. as members. It is possible to inherit: the class can beopenclass
,baseclass
.interfacedef
is provided to define interfaces. Usemethod
to create a signature.
You can use extends
and implements
to implement inheritance.
Note: Rdgen
can generate models for C# and C++. Their structs and classes have different behavior.
Rd entities — only in bindable models (Ext
, classdef
):
property
signal
source
sink
array
andimmutablelist
Useful properties in DSL entities:
async
— the same asasync
in Rd entitiesdocs
— provides KDoc/Javadoc documentation comments for the generated entity
RdGenExtension
configures Rdgen
. The properties are:
sources
— the folders with DSL.kt
files. If there are nosources
, scan classpath for the inheritors ofRoot
andExt
.hashfile
— a folder to store the.rdgen
hash file for incremental generation.packages
— Java package names to search in toplevels, delimited by,
. Example:com.jetbrains.rd.model.nova,com, org
.
Configure model generation with the RdGenExtension.generator
method:
root
— for which root this generator is declared.namespace
— which namespace should be used in the generated source. In Kotlin, it configures the generated package name.directory
— where to put the generated files.transform
— can besymmetric
,asis
, andreversed
. It allows configuring of different model interfaces for various client-server scenarios. Note: in 99% of cases you should usesymmetric
. If you need another option, consult with someone.language
— can bekotlin
,cpp
orcsharp
.
The utbot-rd
Gradle project contains model sources in rdgenModels
. You can find them at
utbot-rd/src/main/rdgen/org/utbot/rd/models
.
An IDE process uses bundled JetBrains JDK. Code in utbot-intellij
must be compatible will all JDKs and plugin
SDKs, used by our officially supported Intellij IDEA versions.
See sinceBuild
and untilBuild
in utbot-intellij/build.gradle.kts
.
The IDE process starts the Engine process. The IDE process keeps the UtSettings
instance in memory and gets updates for it from Intellij IDEA. The other processes "ask" the IDE process about settings via Rd RPC.
TestCaseGenerator
and UtBotSymbolicEngine
run here, in the Engine process. The process classpath contains all
the plugin JAR files (it uses the plugin classpath).
The Engine process must run on the JDK that is used in the project under analysis. Otherwise, there will be
numerous problems with code analysis, soot
, Reflection, and the divergence of the generated code Java API will occur.
Currently, it is prohibited to run more than one generation process simultaneously (the limitation is related to the characteristics of the native libraries). The process logging mechanism relies on that fact, so UnitTestBot processes can exclusively write to a log file.
The starting point in the IDE process is the
EngineProcess
class.
The Engine process start file is
EngineProcessMain
.
The Engine process starts the Instrumented process.
The starting points in the Engine process are the
InstrumentedProcess
and the ConcreteExecutor
classes. The first one encapsulates the state, while the second one implements the request logic for concrete execution.
The Instrumented process runs on the same JDK as the Engine process to prevent deviation from the Engine process. Sometimes the Instrumented process may unexpectedly die due to concrete execution.
- If you need to use Rd, add the following dependencies:
implementation group: 'com.jetbrains.rd', name: 'rd-framework', version: rdVersion implementation group: 'com.jetbrains.rd', name: 'rd-core', version: rdVersion
- There are useful classes in
utbot-rd
to work with Rd and processes:LifetimedProcess
binds aLifetime
instance to a process. If the process dies, theLifetime
instance terminates, and vice versa. You can terminate theLifetime
instance manually — this will destroy the process.ProcessWithRdServer
starts the Rd server and waits for the connection.ClientProtocolBuilder
— you can use it in a client process to correctly connect toProcessWithRdServer
.
- How
ProcessWithRdServer
communication works:- Choose a free port.
- Create a client process and pass the port as an argument.
- Both processes create protocols, bind the model and setup callbacks.
- A server process cannot send messages until the child creates a protocol (otherwise, messages are lost), so the client process has to signal that it is ready.
- The client process creates a special file in the
temp
directory, which is observed by a parent process. - When the parent process spots the file, it deletes this file and sends a special message to the client process confirming communication success.
- Only when the answer of the client process reaches the server, the processes are ready.
- How to write custom RPC commands:
- Add a new
call
in a model, for example, inEngineProcessModel
. - Re-generate models: there are special Gradle tasks for this in the
utbot-rd/build.gradle
file. - Add a callback for the new
call
in the corresponding start files, for example, inEngineProcessMain.kt
. - Important: do not add
Rdgen
as an implementation dependency — it breaks some JAR files as it containskotlin-compiler-embeddable
.
- Add a new
- Logging & debugging:
- Custom protocol marshaling types: do not spend time on it until
UtModels
get simpler, e.g. compatible withkotlinx.serialization
.