Runtime Integration: Alternative to the fat birt_runtime*.jar?

[patric-r]

Question here.

To date, we integrate the BIRT runtime into our application by using the BIRT Runtime zip artifact.
This zip contains several jars as regular dependencies, however interestingly, also a jar named org.eclipse.birt.runtime_4.x.jar.

This jar contains many further dependencies in the good old "fat jar" manner, like guava, slf4j, cassandra and many more.
And because of the hidden nature of fat-jars, it is cumbersome to resolve conflicts with other dependencies of our software.
Currently, we do this by deleting stuff in the jar which we assume are not required but of course, this introduces some risks.

Is there an alternative for this approach? And if not, can we do better than that?

Ideally, BIRT would provide runtime artifacts which do not contain any fat jars.
And even better, the build artifact or its documentation would express which dependencies are optional
and safe to exclude (e.g. you only need the POI jars when you want to use the office emitters) or if required, which version ranges are acceptable in order to provide maximum compatibility and to enable the safe usage of jar-dependencies we provide (which has additional benefits like security aspects).

[hvbtup]

This issue has two components:

1. Today, any library which is not needed is also not wanted, for security reasons.
2. You want to ged rid of the fat jars.

The first component is related to #1217, #1037, #1033 and #951.

In my comments to #951, I showed how I got rid of all other DB libraries except Oracle (which we need in our use-case) by modifying the source projects.

And @wimjongman said that it would be nice if we could get drivers dynamically.

If we transmit Wim's idea also to the emitters, we would have a relatively small BIRT core, and a bunch of input drivers (read: DB drivers / support libraries) and emitters (read: supported output formats).
BIRT users would then be able to create a customized BIRT by downloading the BIRT core and only those DB drivers and emitters they need.

This would solve point 1). However, I have no clue how to achieve this technically. Still being an OSGI noob, my gut feeling that it should be possible with the feature/bundle/plugin techniques.

I can't say anything about point 2) and I don't know if 1) and 2) can be handled independently.

[patric-r]

Hi @hvbtup,

Your comments in issue #951 matches more or less our expectations (beside the fat jar aspect).
Sorry that I've missed this issue.

The "minimalistic core" with optional "plugins" is exactly what we need and probably others would need as well.

When it comes to JDBC - are there any hard-wired references to driver specifics (e.g. classes)?
If yes, can't we refactor those to use pure JDBC instead (e.g. by using the DriverManager for obtaining a connection)?
If not, why are those packaged into the birt runtime jar then - why can't those reside in seperate (optional) jars?

The emitter plugin idea sounds excellent, too.

We're neither using OSGI (so we're "OSGI noobs" as well :) ) therefore we would need a non-OSGI solution.
I don't think that there are many OSGI users out there when it comes to BIRT runtime integration (except for the designer itself).

Regarding guava - ideally BIRT should either get rid of guava or repackage it to avoid any conflicts for non-OSGI integration scenarios. I didn't find any explicit guava usages in BIRT - is guava required because of a transitive dependency?

IMHO, resolving these issues should be a top-priority task as so many BIRT users would probably benefit from it.

[hvbtup]

As I said in #951, I'm not able to to this myself. Creating a proper concept for this kind of modularizazion and coding it requires someone with better Eclipse know-how than me.

[siepkes]

I don't think that there are many OSGI users out there when it comes to BIRT runtime integration

We use BIRT embedded in our application which is an OSGi environment. Ironically we don't use the birt-runtime-osgi artifact but we embed the birt-runtime JAR in a reporting bundle of our own. The reason for this is that the OSGi runtime creates dependency hell within OSGi for us. So by embedding the normal runtime in a bundle we shield the rest of the OSGi environment from the BIRT deps. Now obviously this is very "un-OSGi". But with a solution as big as BIRT which pulls in so many dependencies it's very hard to manage. We did do it for a while like this, but there is really very little incentive to do so and very real pain ;-).

[wimjongman]

That sounds great to me. What other dependencies do you add to that bundle? Can you make a small application that uses this strategy and post the zip here?

[siepkes]

Would love to but creating a simple PoC with OSGi is kinda hard because you also need to setup an OSGi runtime environment. It's more the kind of thing you need a full blog post for.

Currently we use the Felix Bundle plugin to create the OSGi-fied manifest and use pax-runner to start during runtime. We are now using BIRT 4.4 and I'm actually in the process of upgrading it.

The instructions below include all (transitive) dependencies which BIRT needs in to the bundle's JAR and only the bundle classloader is exposed to them. Our reporting bundle then exposes a much simpler API to the rest of the OSGi application.

I don't actually know if the fakeplugin workaround is still needed. It might be a left over from the time we used the actual OSGi BIRT runtime.

<!-- CUT FOR BREVITY -->

<depedencies>
  <!-- BIRT runtime dependencies to be embedded -->
  <dependency>
    <groupId>org.eclipse.birt.runtime</groupId>
    <artifactId>org.eclipse.birt.runtime</artifactId>
    <version>${birt.version}</version>
  </dependency>
  </dependency>
  <!-- Adding this dependency, which ends up as the first bundle on the bundle-classpath, fixes birt 
plugin.xml loading, because it finds this bundles plugin.xml for each bundle without one. Removing this 
dependency results in birt finding another plugin.xml (next on the bundle-classpath), which results 
in duplicate extensions being registered, which breaks aggregation and most graphs. -->
  <dependency>
    <groupId>nl.serviceplanet</groupId>
    <artifactId>eclipse.fakeplugin</artifactId>
    <version>1.0.0</version>
  </dependency>
</depedencies>

<!-- CUT FOR BREVITY -->

<plugin>
  <groupId>org.apache.felix</groupId>
  <artifactId>maven-bundle-plugin</artifactId>
  <extensions>true</extensions>
  <!-- CUT FOR BREVITY -->
  <configuration>
    <!-- CUT FOR BREVITY -->
    <instructions>
      <!-- CUT FOR BREVITY -->
      <Embed-Transitive>true</Embed-Transitive>
      <!-- Make sure eclipse.fakeplugin is the first embedded dependency on the classpath (see comments near dependency declaration) -->
      <Embed-Dependency>eclipse.fakeplugin,*;scope=compile|runtime</Embed-Dependency>

[hvbtup]

Hmm... Jasper's approach shows how one can avoid the dependency hell and how to avoid conflicts for an OSGI environment, but I think that should be moved to a separate discussion, because it's only loosely related to the OP's issue, which was about fat jars, depency hell and conflicts in the POJO runtime and if we can find a way to avoid delivering many unneeded libraries.
E.g. an application that uses BIRT for database reporting (which I assume is the typical use-case) only needs support for this type of database and not for several other DBs, the same holds for emitters.

[hvbtup]

@patric-r Is it ok for you to close this issue and continue in #951?

[patric-r]

@hvbtup For me it's okay but I don't want to cut any discussions - feel free to close it.

BTW, we have a full decoupling from our birt reports to our database and only use input files to propagate data.
Therefore, our ideal BIRT runtime would contain no db driver at all.

[speckyspooky]

The mentioned issue-ticked #951 will be fixed with BIRT 4.14.
The Cassandra Scripted Data Source incl. Hector-runtime is removed with #1419.
So also guava and SLF4J which was included there will be removed.

[patric-r]

That are great news. Thank you, @speckyspooky
Looking forward to see a more lightweight BIRT :)

[eureka-bpo]

Here we have prepared this artifact for usage in Maven project. For our needs it is enough. May be it will useful for you too