Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JaCoCo Support? #529

Closed
jonathanamartin opened this issue Dec 11, 2017 · 64 comments
Closed

JaCoCo Support? #529

jonathanamartin opened this issue Dec 11, 2017 · 64 comments
Assignees
Milestone

Comments

@jonathanamartin
Copy link

Is JaCoCo support on the roadmap for the Java Buildpack?

@nebhale nebhale self-assigned this Dec 11, 2017
@nebhale
Copy link
Contributor

nebhale commented Dec 11, 2017

It is not. The buildpack is responsible for setting up the execution environment for an application, generally in production or QA workloads. Integrating JaCoCo, a development tool, into the buildpack doesn't seem to be a good fit with that goal.

What use case are you trying to solve by integrating it into the buildpack instead of into your application's build?

@jonathanamartin
Copy link
Author

We use JaCoCo for test coverage metrics for unit tests at build time. We also do integration tests on deployed apps. JaCoCo provides test coverage for the integration tests performed against the running app.

To do this, we're currently embedding the JaCoco Agent in the app then referencing it via JAVA_OPTS.

e.g

-javaagent:[yourpath/]jacocoagent.jar=[option1]=[value1],[option2]=[value2]

Would it make sense to manage the JaCoCo agent as a framework in the buildpack? Or would this be something we should manage ourselves say in the resources directory?

@nebhale
Copy link
Contributor

nebhale commented Dec 12, 2017

How do you get data out of the agent and back to a system to visualize it?

@jonathanamartin
Copy link
Author

jonathanamartin commented Dec 12, 2017 via email

@nebhale
Copy link
Contributor

nebhale commented Dec 12, 2017

Then this is a reasonable choice for a framework integration. We'd expose that metadata via a service binding (probably a user-provided service in most cases). Why don't you start by putting some more information in this issue with regards to documentation about where the agent lives, what kind of configuration you currently do, documentation about how the agent is configured, etc. Let's use this issue to flesh out what the integration will look like and then we'll get on it.

@nebhale nebhale removed the question label Dec 12, 2017
@nebhale nebhale added this to the v4.8 milestone Dec 12, 2017
@jonathanamartin
Copy link
Author

Sounds good. As mentioned, we're embedding the agent JAR in the app. Here's an example of the options we're using:

JAVA_OPTS: -javaagent:/home/vcap/app/WEB-INF/org.jacoco.agent-0.7.10-runtime.jar=output=tcpclient,address={address},port={port}

However, it may make sense to provide support for other options.

Not sure if this is needed but v 0.7.10 is important for us because it "reduce[s] chance of conflict with other agents"

@nebhale
Copy link
Contributor

nebhale commented Dec 13, 2017

Looking over the list, output, address, and port are obvious choices as you've used them. Any thought on if people commonly used includes and excludes? Also, would it be useful to tie sessionid to something like the instance identifier created by the platform? Finally, since we do allow a user to start JMX access tunneled over an SSH connection into the container, would enabling JMX access be appropriate?

@jonathanamartin
Copy link
Author

Sorry for the delay. We haven't used includes or excludes but I could see use cases where they would apply. Agreed, instance ID as the sessionid makes sense.

@nebhale
Copy link
Contributor

nebhale commented Dec 14, 2017

Great, that should be enough for me to get started.

On another note, there doesn't appear to be a v0.7.10.

@jonathanamartin
Copy link
Author

jonathanamartin commented Dec 14, 2017 via email

nebhale added a commit to cloudfoundry/java-buildpack-dependency-builder that referenced this issue Dec 14, 2017
This change adds support for publishing JaCoCo agent artifacts.

[cloudfoundry/java-buildpack#529]
nebhale added a commit that referenced this issue Dec 14, 2017
This change updates the buildpack to include JaCoCo agent support.

[resolves #529]
@nebhale
Copy link
Contributor

nebhale commented Dec 14, 2017

@jonathanamartin You should be able to test this now. You can push an application with a bound service as documented and a buildpack specified by -b https://github.com/cloudfoundry/java-buildpack.git#529-jacoco. You should see the agent downloaded during staging and data should stream to it at runtime.

nebhale added a commit that referenced this issue Dec 14, 2017
This change updates the buildpack to include JaCoCo agent support.

[resolves #529]
@jonathanamartin
Copy link
Author

Thanks @nebhale. I'm in an environment that has fairly stringent network policies so I'll have to see what I can do to test. I'll keep you posted.

@jonathanamartin
Copy link
Author

@nebhale I've been discussing the usage of JaCoCo as a means to analyze integration test coverage at runtime with a few colleagues where I work. There has been discussion about dev/prod parity and that we would never want JaCoCo to be used in production.

That said, we could blacklist its usage through our CI/CD pipelines. However, given that we would never use it in production do you still think it's a good candidate for a framework in the buildpack?

@nebhale
Copy link
Contributor

nebhale commented Dec 15, 2017

You seemed to think so earlier, so there's at least someone who thinks its a good candidate. 😄 We have precedent for things you'd never want to run in production like Debug, JRebel, JMX, and Profiling, so adding dev integrations isn't something we shy away from.

I've got some opinions about whether to integrate it, but not strongly held ones. For something like this, we'd never add it proactively, but someone asked, it doesn't harm other users, and so I've implemented it. If you are having second thoughts, there's no problem not adding it at this point.

@bhanafee
Copy link

Since there are other dev integrations, adding JaCoCo support makes sense. It is widely used, and there do exist system and integration test cases where it would be useful to gather coverage data.

@jonathanamartin
Copy link
Author

@nebhale and @bhanafee thanks for the feedback. @nebhale let's proceed. I'll do the validation you had requested. Thanks for the outstanding support!

@nebhale
Copy link
Contributor

nebhale commented Dec 19, 2017

Great. I look forward to confirmation.

@tdrury
Copy link

tdrury commented Dec 19, 2017

is it possible to reference this jacoco-enabled buildpack in an MTA yaml?

@nebhale
Copy link
Contributor

nebhale commented Dec 19, 2017

I’m not an expert in SAP’s CF distribution, but since it is a certified distribution it’s supposed to be able to run all official buildpacks. As this is a branch of the official Java Buildpack, I’d expect it to work. Give it a try and let me know 😄

@tdrury
Copy link

tdrury commented Dec 19, 2017

I want to try, but I'm unsure of the syntax of the mtad.yaml file to use a buildpack from a URL instead of:
type: java.tomcat

@tdrury
Copy link

tdrury commented Dec 19, 2017

is specifying the buildpack the only thing I need? I managed to deploy an MTA with the buildpack URL of the 529-jacoco branch, but I don't see "jacoco" in the VCAP_SERVICES of the Java app.

@nebhale
Copy link
Contributor

nebhale commented Dec 19, 2017

Buildpack syntax can take the form of a URI. So you can specify -b http://github.com/cloudfoundry/java-buildpack.git#529-jacoco or the manifest equivalent of the same.

@ishaan2007
Copy link

we will be able to override /specify sessionid correct?

@xunwu
Copy link

xunwu commented Jan 26, 2018

Hi All,
I have pushed an app to the cloud foundry with this "-b http://github.com/cloudfoundry/java-buildpack.git#529-jacoco" option and I can see my app successfully started, but I cannot see any logs related to jacoco, I am wondering is there any other configuration I need to do for the jacoco to generate the coverage report, for example specify JAVA_OPTS related to jacoco?

@nebhale
Copy link
Contributor

nebhale commented Jan 29, 2018

@xunwu Please see the documentation

@xunwu
Copy link

xunwu commented Jan 29, 2018

@nebhale So do you mean that I should create a jacoco service and bind it to my application? I cannot find any jacoco service in marketplace, how can I create one?

@shinmyung0
Copy link

@xunwu I think you should be creating a User-Provided service with the appropriate credentials

@nebhale
Copy link
Contributor

nebhale commented Jan 29, 2018

The documentation describes the payload for a user-provided service. You should use one of those.

@xunwu
Copy link

xunwu commented Jan 29, 2018

@nebhale Sorry I am new to user-provided service, but do I need to create a user-provided service by using " cf cups my-jacoco-service -p '{"address":"127.0.0.1","includes":"*", "port":"1010"}' " but what address and port should I use, is it the address for some jacoco microservice running on the cloud foundry?

@xunwu
Copy link

xunwu commented Jan 29, 2018

I used this option and see Jacoco being downloaded
-----> Downloading Jacoco Agent 0.8.0 from https://java-buildpack.cloudfoundry.org/jacoco/jacoco-0.8.0.jar (1.0s)
Expanding Jacoco Agent to .java-buildpack/jacoco_agent (0.0s)

name: xun-javaexample
requested state: started
instances: 1/1
usage: 928M x 1 instances
routes: xun-javaexample-introspective-ovolo.run.aws-usw02-pr.ice.predix.io
last uploaded: Mon 29 Jan 09:55:49 PST 2018
stack: cflinuxfs2
buildpack: http://github.com/cloudfoundry/java-buildpack.git#529-jacoco
start command: JAVA_OPTS="-agentpath:$PWD/.java-buildpack/open_jdk_jre/bin/jvmkill-1.12.0_RELEASE=printHeapHistogram=1 -Djava.io.tmpdir=$TMPDIR
-javaagent:$PWD/.java-buildpack/jacoco_agent/jacocoagent.jar=address=127.0.0.1,output=tcpclient,sessionid=$CF_INSTANCE_ID,includes=*,port=1010
-Djava.ext.dirs=$PWD/.java-buildpack/container_security_provider:$PWD/.java-buildpack/open_jdk_jre/lib/ext -Djava.security.properties=$PWD/.java-buildpack/java_security/java.security
$JAVA_OPTS" && CALCULATED_MEMORY=$($PWD/.java-buildpack/open_jdk_jre/bin/java-buildpack-memory-calculator-3.10.0_RELEASE -totMemory=$MEMORY_LIMIT -stackThreads=250 -loadedClasses=13289
-poolType=metaspace -vmOptions="$JAVA_OPTS") && echo JVM Memory Configuration: $CALCULATED_MEMORY && JAVA_OPTS="$JAVA_OPTS $CALCULATED_MEMORY" && SERVER_PORT=$PORT JAVA_OPTS=$JAVA_OPTS
JAVA_HOME=$PWD/.java-buildpack/open_jdk_jre exec $PWD/java-hello-world-cf-example/bin/java-hello-world-cf-example

So how can I grab the jacoco data on the fly?

@shinmyung0
Copy link

shinmyung0 commented Jan 29, 2018

@nebhale This was one thing I was also wondering, If I were to deploy the receiving service on PCF itself can it be something like a go-route with port 80 or 443? Or does it need to be a tcp route?

@xunwu
Copy link

xunwu commented Jan 29, 2018

@shinmyung0 if you look at here http://www.eclemma.org/jacoco/trunk/doc/agent.html
the jacoco report can only dump through TCP Connection, I am also wondering how do you trigger jacoco to generate the report

@xunwu
Copy link

xunwu commented Feb 6, 2018

I was trying this Jacoco build pack for a while but even though I got this configuration set up in cloud foundry, and see that it has the same JAVA_OPTS as in my local, I am not able to dump the file, I set up the jacoco to dump through tcp connection and use localhost and CF_INSTANCE_PORT to dump the report on the same instance, but it got an error for java.io.IOException: Invalid execution data file.so that means cloud foundry is not allowing jacoco agent to export data? I looked at the source code, which means the output stream is not in correct format or it could be an empty output stream, I am little confused, so cloud foundry is not allowing any other process to use any port in that container?

@nebhale
Copy link
Contributor

nebhale commented Feb 12, 2018

@jonathanamartin Did you ever get a chance to test this?

@jonathanamartin
Copy link
Author

@nebhale I apologize, facing a few roadbloack on my end and haven't been able to take the testing any further. Though, I have something in mind and will get back with you in a day or so. Thank you for your patience.

@nebhale
Copy link
Contributor

nebhale commented Feb 13, 2018

No need to apologize. I'm buttoning down v4.9 now and want to get it in if possible. If not, it'll just slide out to v4.10.

@Aeolic
Copy link

Aeolic commented Feb 14, 2018

Hi @nebhale, I'm currently using the jacoco Buildpack and it works great - the only problem right now (really minor) is that I don't get any Session IDs, the applications are started with "sessionid=$CF_INSTANCE_ID" as intended, but the session IDs are just blank - has anyone else experienced this, or is this more likely to be a mistake on our side? Thanks!

@nebhale
Copy link
Contributor

nebhale commented Feb 14, 2018

@Aeolic Could simply be an escaping problem. Any chance you can send over the command line (as reported by the CLI) that is run when you're successfully sending data?

@Aeolic
Copy link

Aeolic commented Feb 15, 2018

@nebhale I might misunderstand what you mean, but the command line never shows anything about jacoco sending data (I trigger a dump by using a custom TCP server and send a "dump" command to the jacoco client) - another question: Is it possible to supply port and address without using UPSs ? We have quite a lot of microservices running and right now I have to create a config for each service, because I want to use a different port for each service - Thanks!

@nebhale
Copy link
Contributor

nebhale commented Feb 15, 2018

@Aeolic When the buildpack notices a jacoco service, it adds the appropriate -javaagent to the command line:

-javaagent:$PWD/.java-buildpack/jacoco_agent/jacocoagent.jar=address=test-address,output=tcpclient,sessionid=$CF_INSTANCE_ID,excludes=test-excludes,includes=test-includes,port=6300

This is how the agent knows where to send data. I suspect that the problem is that the $CF_INSTANCE_ID is being evaluated too early and needs to be escaped, but I'm looking for some confirmation of that since I'm doing this a bit blind.

In 12 Factor applications, (CF is fundamentally a system for running 12 Factor applications) connections to external system are provided by the environment. In CF, the manifestation of this idea is the concept of services. So whether it's a database, an APM, or even a code coverage sink, all information of where to connect to and the credentials for connecting with are communicated via services. From the perspective of the buildpack we don't care if a service is a user-provided service or a service-broker (i.e. "managed") service, so we always start with instructions on how to use a User Provided Service to communicate this. If you find that it's very useful, you should absolutely write a Service Broker that knows how to spin up instances of the JaCoCo servers when cf create-service is called and provide that information to applications when cf bind-service is called.

@Aeolic
Copy link

Aeolic commented Feb 16, 2018

@nebhale now I understand what you were talking about, the part "when you're successfully sending data" got me a bit confused :)

JAVA_OPTS="-agentpath:$PWD/.java-buildpack/open_jdk_jre/bin/jvmkill-1.12.0_RELEASE=printHeapHistogram=1 -Djava.io.tmpdir=$TMPDIR -javaagent:$PWD/.java-buildpack/jacoco_agent/jacocoagent.jar=address=someIP,output=tcpclient,sessionid=$CF_INSTANCE_ID,port=somePort -Djava.ext.dirs=$PWD/.java-buildpack/container_security_provider:$PWD/.java-buildpack/open_jdk_jre/lib/ext -Djava.security.properties=$PWD/.java-buildpack/java_security/java.security $JAVA_OPTS" && CALCULATED_MEMORY=$($PWD/.java-buildpack/open_jdk_jre/bin/java-buildpack-memory-calculator-3.10.0_RELEASE -totMemory=$MEMORY_LIMIT -stackThreads=250 -loadedClasses=14041 -poolType=metaspace -vmOptions="$JAVA_OPTS") && echo JVM Memory Configuration: $CALCULATED_MEMORY && JAVA_OPTS="$JAVA_OPTS $CALCULATED_MEMORY" && SERVER_PORT=$PORT eval exec $PWD/.java-buildpack/open_jdk_jre/bin/java $JAVA_OPTS -cp $PWD/. org.springframework.boot.loader.JarLauncher

This is the command line we are using to launch the apps.

And thanks for the other answer as well!

@nebhale
Copy link
Contributor

nebhale commented Feb 16, 2018

OK, the problem is that the variable is called $CF_INSTANCE_GUID not $CF_INSTANCE_ID. Updated now, so @Aeolic please have another go and let me know if you're getting a proper sessionid.

@Aeolic
Copy link

Aeolic commented Feb 19, 2018

@nebhale I'm getting a id now, works great! Thanks so much for your help!

@jonathanamartin
Copy link
Author

This is somewhat unconventional. However, @nebhale if you're good with the testing @Aeolic has completed I am too :).

@nebhale
Copy link
Contributor

nebhale commented Feb 20, 2018

I'm heading off to a conference for the rest of the week and would really like to get this merged this morning for a release, so I'm going to call this confirmed. We can always change it if there's something wrong, but I think this is sufficient for a first try.

@jonathanamartin
Copy link
Author

Thanks!

@tdrury
Copy link

tdrury commented Mar 22, 2018

I have a question about getting the coverage data out of CF: um, how do you do it? I've always used tcpserver mode on my (non-cloud) servers and it's fairly easy.

Is it possible to configure jacoco for tcpserver mode or is only tcpclient allowed?

I've never used tcpclient mode. When does it send coverage information? Reading the docs on tcpclient, it's not clear.

@konstantinblaesi
Copy link
Contributor

konstantinblaesi commented May 14, 2019

For another option see #715 (comment)

@pank9
Copy link

pank9 commented Nov 3, 2020

I used this option and see Jacoco being downloaded
-----> Downloading Jacoco Agent 0.8.0 from https://java-buildpack.cloudfoundry.org/jacoco/jacoco-0.8.0.jar (1.0s)
Expanding Jacoco Agent to .java-buildpack/jacoco_agent (0.0s)

name: xun-javaexample
requested state: started
instances: 1/1
usage: 928M x 1 instances
routes: xun-javaexample-introspective-ovolo.run.aws-usw02-pr.ice.predix.io
last uploaded: Mon 29 Jan 09:55:49 PST 2018
stack: cflinuxfs2
buildpack: http://github.com/cloudfoundry/java-buildpack.git#529-jacoco
start command: JAVA_OPTS="-agentpath:$PWD/.java-buildpack/open_jdk_jre/bin/jvmkill-1.12.0_RELEASE=printHeapHistogram=1 -Djava.io.tmpdir=$TMPDIR
-javaagent:$PWD/.java-buildpack/jacoco_agent/jacocoagent.jar=address=127.0.0.1,output=tcpclient,sessionid=$CF_INSTANCE_ID,includes=*,port=1010
-Djava.ext.dirs=$PWD/.java-buildpack/container_security_provider:$PWD/.java-buildpack/open_jdk_jre/lib/ext -Djava.security.properties=$PWD/.java-buildpack/java_security/java.security
$JAVA_OPTS" && CALCULATED_MEMORY=$($PWD/.java-buildpack/open_jdk_jre/bin/java-buildpack-memory-calculator-3.10.0_RELEASE -totMemory=$MEMORY_LIMIT -stackThreads=250 -loadedClasses=13289
-poolType=metaspace -vmOptions="$JAVA_OPTS") && echo JVM Memory Configuration: $CALCULATED_MEMORY && JAVA_OPTS="$JAVA_OPTS $CALCULATED_MEMORY" && SERVER_PORT=$PORT JAVA_OPTS=$JAVA_OPTS
JAVA_HOME=$PWD/.java-buildpack/open_jdk_jre exec $PWD/java-hello-world-cf-example/bin/java-hello-world-cf-example

So how can I grab the jacoco data on the fly?

Hi

Hi All,
I have pushed an app to the cloud foundry with this "-b http://github.com/cloudfoundry/java-buildpack.git#529-jacoco" option and I can see my app successfully started, but I cannot see any logs related to jacoco, I am wondering is there any other configuration I need to do for the jacoco to generate the coverage report, for example specify JAVA_OPTS related to jacoco?

Hi Xunwu,
I am trying to deploy the application from my command line using below command in order to use jacoco agent support but it is not working. I am getting error "ERR Failed to clone git repository at https://github.com/cloudfoundry/java-buildpack.git"

Command
cf push coverage-repo1 -b https://github.com/cloudfoundry/java-buildpack.git#529-jacoco

I am deploying app on cloud foundry with jacoco agent support to generate test case report.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests