Skip to content

Tutorial: pankti

Jump to bottom
Deepika Tiwari edited this page Jun 4, 2022 · 3 revisions

A tutorial on test generation with pankti

Step 1: Setting up

pankti

First, build pankti

cd pankti/
mvn clean install

A case study: BroadLeaf

For these tutorials, we work with an open-source Java project called BroadLeafCommerce. Let's start by cloning it.

git clone [email protected]:BroadleafCommerce/BroadleafCommerce.git --branch broadleaf-6.2.0-GA

The developers of BroadLeafCommerce also provide a demo application called DemoSite. Let's clone that as well. We will use DemoSite for our experiments.

git clone [email protected]:BroadleafCommerce/DemoSite.git

You can build and run it, following the steps in its README. Play around with the website (localhost:8443) to get familiar with it.

Glowroot

Glowroot is an open-source APM agent. We use it for instrumentation and monitoring. Follow the instructions on the website to download it. Also, create a directory called plugins within glowroot/.

You can try to attach glowroot as a javaagent to DemoSite, by updating its POM and then running DemoSite.

...
    <properties>
      <debug.args>-Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=${debug.port}</debug.args>
      <maven.compiler.source>11</maven.compiler.source>
      <maven.compiler.target>11</maven.compiler.target>
      <boot.jvm.memory>-Xmx1536M</boot.jvm.memory>
-     <boot.jvm.args>${boot.jvm.memory} ${debug.args}</boot.jvm.args>
+     <glowroot.plugin.jar>-javaagent:/path/to/glowroot/glowroot.jar</glowroot.plugin.jar>
+     <boot.jvm.args>${boot.jvm.memory} ${debug.args} ${glowroot.plugin.jar}</boot.jvm.args>
      <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
      <project.uri>${project.baseUri}</project.uri>
    </properties>
...

Step 2: Extracting a list of target methods

Next, we statically analyze the BroadLeafCommerce source to find public methods that can be our targets for test generation.

cd pankti-extract/
java -jar target/pankti-extract-<version>-jar-with-dependencies.jar /path/to/BroadleafCommerce/

This produces ./extracted-methods-BroadLeafCommerce.csv

Step 3: Instrumenting the target methods

Next, we instrument these methods so that the object they are called on (receiving object), the objects passed as parameters, as well as the object returned from this method, are serialized as DemoSite executes. First,

cd pankti-instrument/

Since the ./extracted-methods-BroadLeafCommerce.csv we created in Step 2 has a lot of methods, we can find a list of classes with the most methods with the following:

python3 find-cuts.py ../pankti-extract/extracted-methods-BroadLeafCommerce.csv

This creates two files: ./cuts.txt and ./cuts-mockables.txt. We'll use cuts.txt for test generation with pankti. It has a list of classes, as well the number of target methods within them. Let's pick org.broadleafcommerce.core.catalog.service.dynamic.DynamicSkuPrices which has 6 potential target methods that we can instrument.

python3 instrument.py ../pankti-extract/extracted-methods-BroadLeafCommerce.csv  org.broadleafcommerce.core.catalog.service.dynamic.DynamicSkuPrices

This generates 6 aspect classes (MethodAspect1 to MethodAspect6) in se.kth.castor.pankti.instrument.plugins and also updates pankti-instrument/src/main/resources/META-INF/glowroot.plugin.json with this list of new aspects. We can finally package this into a plugin with

mvn clean install

This produces pankti-instrument/target/pankti-instrument-<version>-jar-with-dependencies.jar. Copy this jar and paste it in the glowroot/plugins/ directory we created in Step 1.

Step 4: Executing DemoSite and collecting data

Now that we've added the plugin with the serialization instructions to glowroot, run DemoSite (make sure you've modified the DemoSite POM as presented in Step 1). Click around the website and interact with the products, add them to the cart, etc. You'll find that /tmp/pankti-object-data/ has some XML files (which are the serialized versions of objects) and an invoked-methods.csv (which is a list of the methods that were invoked within org.broadleafcommerce.core.catalog.service.dynamic.DynamicSkuPrices as a result of your interactions). You can stop the DemoSite server when you're done interacting with the site.

Step 5: Generating tests

Finally,

cd pankti-generate/
java -jar target/pankti-generate-<version>-jar-with-dependencies.jar /path/to/BroadleafCommerce/ /tmp/pankti-object-data/invoked-methods.csv /tmp/pankti-object-data/

This generates tests within pankti-generate/output/generated/BroadLeafCommerce/ as well as related resource files for longer object XMLs. Find the list of generated test classes with

cd output/generated/BroadLeafCommerce/
find . -name "Test*PanktiGen"

We see that we've generated the test class TestDynamicSkuPricesPanktiGen.java. The generated resource files are in pankti-generate/output/generated/object-data/.

Step 6: Executing the tests

Create a new project module within BroadLeadCommerce called pankti-tests, add the following properties and dependencies in its POM.

  <properties>
    <maven.compiler.source>11</maven.compiler.source>
    <maven.compiler.target>11</maven.compiler.target>
    <junit.jupiter.engine.version>5.8.2</junit.jupiter.engine.version>
    <junit.platform.runner.version>1.8.2</junit.platform.runner.version>
    <junit.jupiter.params.version>5.8.2</junit.jupiter.params.version>
    <xstream.version>1.4.12</xstream.version>
  </properties>

  <dependencies>
    <dependency>
      <groupId>com.thoughtworks.xstream</groupId>
      <artifactId>xstream</artifactId>
      <version>${xstream.version}</version>
    </dependency>
    <dependency>
      <groupId>org.junit.jupiter</groupId>
      <artifactId>junit-jupiter-engine</artifactId>
      <version>${junit.jupiter.engine.version}</version>
    </dependency>
    <dependency>
      <groupId>org.junit.platform</groupId>
      <artifactId>junit-platform-runner</artifactId>
      <version>${junit.platform.runner.version}</version>
    </dependency>
    <dependency>
      <groupId>org.junit.jupiter</groupId>
      <artifactId>junit-jupiter-params</artifactId>
      <version>${junit.jupiter.params.version}</version>
     </dependency>
     <dependency>
       <groupId>org.junit.jupiter</groupId>
       <artifactId>junit-jupiter</artifactId>
       <version>${junit.jupiter.engine.version}</version>
     </dependency>
     <dependency>
       <groupId>org.junit.jupiter</groupId>
       <artifactId>junit-jupiter-api</artifactId>
       <version>${junit.jupiter.engine.version}</version>
     </dependency>
     <dependency>
      <groupId>org.broadleafcommerce</groupId>
      <artifactId>broadleaf-framework</artifactId>
      <scope>test</scope>
     </dependency>
  </dependencies>

Create the package org.broadleafcommerce.core.catalog.service.dynamic under src/test/java within this new module. Copy the generated test inside this package. Also create src/test/resources/ and add the generated resource files. The generated tests are now ready to run!

Clone this wiki locally