Using Elementary

⚠️ This wiki is no longer maintained. Please refer to https://github.com/Pante/elementary/tree/master/docs instead.

This article describes how to use Elementary to unit test an annotation processor. We assume that the reader is acquainted with annotation processing and the javax.lang.model.* packages.

If you learn best by jumping straight into code, check out the Gradle demo project contributed by ToolForger!

Tests are required to be compiled against Java 11 although the classes under test can be compiled against earlier version of Java. In addition, the library requires a minimum JUnit version of 5.7.1.

Do join Karus Labs' discord if you require any assistance.

This article details how to use Elementary, if you are interested in how Elementary works, please read The Problem With Annotation Processors.

A Brief Tour of Elementary

At the heart of Elementary is the standalone compiler upon which everything else is built, including the JavacExtension and ToolsExtension JUnit extensions. Configuration of said extensions is done through annotations on the test classes and methods. In the interest of keeping things short and sweet, we shall skim over the standalone compiler since it is seldom used barring a few advanced cases. Most will find the higher-level JavacExtension and ToolsExtension more pleasant to use anyways.

Downloading Elementary

Elementary is available as a maven artifact.

<repository>
  <id>elementary-releases</id>
  <url>https://repo.karuslabs.com/repository/elementary-releases/</url>
</repository>

<!-- Requires JUnit 5.7.1 & above -->
<dependency>
    <groupId>org.junit.jupiter</groupId>
    <artifactId>junit-jupiter-api</artifactId>
    <version>5.7.1</version>
    <scope>test</scope>
</dependency>

<dependency>
    <groupId>com.karuslabs</groupId>
    <artifactId>elementary</artifactId>
    <version>1.1.1</version>
    <scope>test</scope>
</dependency>

The JavacExtension

For each test, JavacExtension compiles a suite of test files with the given annotation processor(s). The results of the compilation are then funneled to the test method for subsequent assertions. All configuration is handled via annotations with no additional set-up or tear-down required.

We recommend using the JavacExtension in the following scenarios:

• Black-box testing an annotation processor
• Testing the results of a compilation
• Testing an extremely simple annotation processor

A typical usage of the JavacExtension will look similar to the following:

import com.karuslabs.elementary.Results;
import com.karuslabs.elementary.junit.JavacExtension;
import com.karuslabs.elementary.junit.annotations.Case;
import com.karuslabs.elementary.junit.annotations.Classpath;
import com.karuslabs.elementary.junit.annotations.Options;
import com.karuslabs.elementary.junit.annotations.Processors;

@ExtendWith(JavacExtension.class)
@Options("-Werror")
@Processors({ImaginaryProcessor.class})
@Classpath("my.package.ValidCase")
class ImaginaryTest {
    @Test
    void process_string_field(Results results) {
        assertEquals(0, results.find().errors().count());
    }
    
    @Test
    @Classpath("my.package.InvalidCase")
    void process_int_field(Results results) {
        assertEquals(1, results.find().errors().contains("Element is not a string").count());
    }
}

@SupportedAnnotationTypes({"*"})
class ImaginaryProcessor extends AnnotationProcessor {
    @Override
    public boolean process(Set<? extends TypeElement> set, RoundEnvironment round) {
        var elements = round.getElementsAnnotatedWith(Case.class);
        for (var element : elements) {
            if (element instanceof VariableElement)) {
                var variable = (VariableElement) element;
                if (!types.isSameType(variable.asType(), types.type(String.class))) {
                    logger.error(element, "Element is not a string");
                }
            } else {
                logger.error(element, "Element is not a variable");
            }
        }
        return false;
    }
}

Let’s break down the code snippet.

• By annotating the test class with @Options, we can specify the compiler flags used when compiling the test cases. In this snippet, -Werror indicates that all warnings will be treated as errors.

• To specify which annotation processor(s) is to be invoked with the compiler, we can annotate the test class with @Processors.

• Test cases can be included for compilation by annotating the test class with either @Classpath, @Resource or @Inline. Java source files on the classpath can be included using @Classpath or @Resource while strings inside @Inline can be transformed into an inline source file for compilation. One difference between @Classpath and @Resource is how directories are separated. @Classpath separates directories using . while @Resource uses /.

• An annotation’s scope is tied to its target’s scope. If a test class is annotated, the annotation will be applied for all test methods in that class. On the same note, an annotation on a test method will only be applied on said method.

• Results represent the results of a compilation. We can specify Results as a parameter of test methods to obtain the compilation results. In this snippet, process_string_field(...) will receive the results for ValidCase while process_int_field(...) will receive the results for both ValidCase and InvalidCase.

Supported Annotations

Annotation	Description	Target
@Classpath	Denotes a class on the current classpath to be included for compilation. Directories are separated by ..	Test class/method
@Inline	A string representation of a class to be included for compilation.	Test class/method
@Options	Represents the compiler flags to be used during compilation.	Test class/method
@Processors	The annotation processors to be applied during compilation.	Test class/method
@Resource	Denotes a file on the current classpath to be included for compilation. Directories are separated by /.	Test class/method

The ToolsExtension

A Java compiler with a blocking annotation processor is invoked on a daemon thread each time an instance of a test class is created. During which, compilation is halted and the annotation processing environment made available to the test. In addition, test cases can be written in plain old Java and its Element representation subsequently retrieved in a test. Similar to JavacExtension, all configuration is handled through annotations.

We recommend using ToolsExtenion in the following circumstances:

• White-box testing an annotation processor
• Testing individual components of an annotation processor
• Testing an annotation processor against multiple complex test cases
• Requiring access to the annotation processing environment

A typical usage of the ToolsExtension will look similar to the following:

import com.karuslabs.elementary.junit.Cases;
import com.karuslabs.elementary.junit.Tools;
import com.karuslabs.elementary.junit.ToolsExtension;
import com.karuslabs.elementary.junit.annotations.Case;
import com.karuslabs.elementary.junit.annotations.Inline;
import com.karuslabs.elementary.junit.annotations.Introspect;
import com.karuslabs.utilitary.type.TypeMirrors;

@ExtendWith(ToolsExtension.class)
@Introspect
@Inline(name = "Samples", source = {
"import com.karuslabs.elementary.junit.annotations.Case;",
"",
"class Samples {",
"  @Case(\"first\") String first;",
"}"})
class ToolsExtensionExampleTest {

    Lint lint = new Lint(Tools.typeMirrors());
    
    @Test
    void lint_string_variable(Cases cases) {
        var first = cases.one("first");
        assertTrue(lint.lint(first));
    }
    
    @Test
    void lint_method_that_returns_string(Cases cases) {
        var second = cases.get(1);
        assertFalse(lint.lint(second));
    }

    @Case String second() { return "";}
    
}

class Lint {
    
    final TypeMirrors types;
    final TypeMirror expectedType;
    
    Lint(TypeMirrors types) {
        this.types = types;
        this.expectedType = types.type(String.class);
    }
    
    public boolean lint(Element element) {
        if (!(element instanceof VariableElement)) {
            return false;
        }
        
        var variable = (VariableElement) element;
        return types.isSameType(expectedType, variable.asType());
    }
    
}

Let’s break down the code snippet:

• Annotating the class with @Introspect includes the test file for compilation. The annotated test class must also be extended with ToolsExtension. An additional name must be specified in the annotation if the annotated class and file is differently named.

• By annotating the class with @Inline we can specify a inline Java source file which ToolsExtension includes for compilation.

• The annotation processing environment can be accessed via either the Tools class or dependency injection into the test class's constructor or test methods.

• By annotating a test case with @Case inside a Java source file, we can fetch it's corresponding element from Cases. A @Case may also contain a label to simplify retrieval. This can be used in conjunction with @Introspect to declare test cases in the test file. By default a @Case annotation with no given label will use its annotation target's name as its label.

• Through Cases, we can fetch elements by either the label or index of the case. We can obtain an instance of Cases via Tools.cases() or through dependency injection.

Supported Annotations

Annotation	Description	Target
@Case	Denotes that the annotated target is a test case that can be retrieved by Cases.	Any test case
@Classpath	Denotes a class on the current classpath to be included for compilation. Directories are separated by ..	Test class
@Inline	A string representation of a class to be included for compilation.	Test class
@Introspect	Includes the test file for compilation. The annotated test class must also be extended with ToolsExtension. An additional name must be specified in the annotation if the annotated class and file is differently named. May require additional configuration, please see @Introspect Configuration.	Test class
@Resource	Denotes a file on the current classpath to be included for compilation. Directories are separated by /.	Test class

Further Reading

Elementary provides two more examples that illustrate how to use JavacExtension and ToolsExtension which may be found here. In addition, Elementary is used in production by Satisfactory to test the multiple assertions that it contains, the tests can be found here.

The Javadocs can be found here.

Parallel JUnit Tests

Elementary does not support parallel testing (at the moment).