Add OSGi manifest headers

[mattbishop]

Those of us on OSGi environments (like eclipse, netbeans, glassfish) cannot use rxjava-core or the other adapters without OSGi headers being added to the manifest.

This is pretty easy to do using the Maven Bundle plugin from apache Felix.

[benjchristensen]

This looks pretty straight-forward to do with Gradle: http://www.gradle.org/docs/current/userguide/osgi_plugin.html

Since I have not used OSGi before, is there anything special about what the headers need to contain or is it basically the same information used to post to Maven Central?

For example, this pom file: http://search.maven.org/#artifactdetails%7Ccom.netflix.rxjava%7Crxjava-core%7C0.5.3%7Cjar

[mattbishop]

This plugin looks similar enough to the felix plugin (which also uses BND) that it should work. The import/exports of RxJava are minimal and discoverable by the java code. There aren't any Class.forName() calls right?

[benjchristensen]

There is one place Class.forName is used – to automatically load the plugins when they are in the classpath:

https://github.com/Netflix/RxJava/blob/master/rxjava-core/src/main/java/rx/util/functions/Functions.java#L49

Language adaptors can also be manually loaded using Functions.registerLanguageAdaptor(Class<?>[] functionClasses, FunctionLanguageAdaptor adaptor)

https://github.com/Netflix/RxJava/blob/master/rxjava-core/src/main/java/rx/util/functions/Functions.java#L63

[mattbishop]

Interesting. This means the language plugins are going to have to be "bundle fragments" instead of plain bundles. That means that the plugin jars will be bound to the main rxjava-core jar at runtime as a logically-merged bundle. I haven't used gradle so my best-guess addendum for the language plugins would be:

instruction 'Fragment-Host', 'com.netflix.rxjava.rxjava-core'

This means the jar being built (like the groovy plugin) is a fragment, whose host is rxjava-core. At runtime OSGi will fit the two together.

[mattbishop]

If you set up the gradle changes I am happy to test.

[benjchristensen]

I'd appreciate that as I have no way to know if it's successful or not, so I'll create a branch with the gradle changes and then if you can play with and modify the values as necessary that would be great. I'm working on something else right now that I need to get done and then can work on this.

[mattbishop]

NP, gives me a chance to experience gradle. :)

On Fri, Feb 22, 2013 at 6:07 PM, Ben Christensen
[email protected]:

I'd appreciate that as I have no way to know if it's successful or not, so
I'll create a branch with the gradle changes and then if you can play with
and modify the values as necessary that would be great. I'm working on
something else right now that I need to get done and then can work on this.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/154#issuecomment-13978672.

[benjchristensen]

I've got a jar building with the OSGi headers but that's where I stop having any idea what to put in as the headers, so I'd appreciate you getting it working as needed and submitting the pull request.

Here is how I did it:

1. Edit build.gradle of the module to change. For example, ./rxjava-core/build.gradle

I added this line at the top:

apply plugin: 'osgi'

and this at the bottom:

jar {
    manifest { 
        name = 'rxjava-core'
        instruction 'Bundle-Vendor', 'Netflix'
        instruction 'Bundle-DocURL', 'https://github.com/Netflix/RxJava'
    }
}

I then run a build:

$ ./gradlew clean build

I then see the MANIFEST file by doing this:

$ cd ./rxjava-core/build/libs/
$ jar xvf rxjava-core-0.5.4-SNAPSHOT.jar 
$ cat META-INF/MANIFEST.MF 

I see the new manifest headers that normally aren't there without the build.grade changes above.

[mattbishop]

Hi Ben, I will give it a go next week.

On Wed, Feb 27, 2013 at 2:49 PM, Ben Christensen
[email protected]:

I've got a jar building with the OSGi headers but that's where I stop
having any idea what to put in as the headers, so I'd appreciate you
getting it working as needed and submitting the pull request.

Here is how I did it:

1. Edit build.gradle of the module to change. For example,
   ./rxjava-core/build.gradle

I added this line at the top:

apply plugin: 'osgi'

and this at the bottom:

jar {
manifest {
name = 'rxjava-core'
instruction 'Bundle-Vendor', 'Netflix'
instruction 'Bundle-DocURL', 'https://github.com/Netflix/RxJava'
}}

I then run a build:

$ ./gradlew clean build

I then see the MANIFEST file by doing this:

$ cd ./rxjava-core/build/libs/
$ jar xvf rxjava-core-0.5.4-SNAPSHOT.jar
$ cat META-INF/MANIFEST.MF

I see the new manifest headers that normally aren't there without the
build.grade changes above.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/154#issuecomment-14205447
.

[mattbishop]

I added in your jar {} section and it does generate OSGi headers in the manifest.

However, the unit tests are in the source code so it inserts dependencies for junit and mockito. I can strip them out with custom BND instructions to the OSGi plugin, but are these tests going to stay in the main src dir or will they move to a separate tests dir?

[mattbishop]

Please add this one line to the jar {} section:

    instruction 'Import-Package', '!org.junit,!junit.framework,!org.mockito.*,*'

This should mask out the unit testing packages.

NOTE: Make sure you set your version to a release name (non-SNAPSHOT) before building the project. The manifest exports specific versions of rx; right now that version is "version="0.5.5.SNAPSHOT" which will fail if the actual maven artifact version is 0.5.5.

Thanks, if you do move the tests out of the main source dir then the Import-Package instruction can be removed.

[benjchristensen]

The unit tests will not be moved out. They are there on purpose.

The JUnit dependency is marked as 'provided' (https://github.com/Netflix/Hystrix/blob/master/hystrix-core/build.gradle#L9) but not included as part of the runtime dependencies when published to Maven Central: http://search.maven.org/#artifactdetails%7Ccom.netflix.hystrix%7Chystrix-core%7C1.2.12%7Cjar

Does the addition of just the following OSGi headers satisfy your requirements?

jar {
    manifest { 
        name = 'rxjava-core'
        instruction 'Bundle-Vendor', 'Netflix'
        instruction 'Bundle-DocURL', 'https://github.com/Netflix/RxJava'
        instruction 'Import-Package', '!org.junit,!junit.framework,!org.mockito.*,*'
    }
}

What about the 'bundle fragments' that you mentioned would be needed for the language-adaptors? Or do I simply just add these same headers to each of those other jars as well (with their appropriate names of course)?

[mattbishop]

The 'provided' scope still means a runtime dependency rather than a test time dependency. I don't understand why they are not separated but that doesn't matter as long as it can be managed out of the dependencies.

As for the language adapters, let me have a look at them. They will need their own instructions for gradle including the fragment instruction.

[mattbishop]

...and to answer the question for core, yes that manifest instruction set is good.

[benjchristensen]

I'll wait on your answer for the language-adaptors and then commit the changes.

Yes, it's "runtime" for running the tests, but not for production usage which is why they are marked as provided and not needed as transitive dependencies. Reasons if you care are here: http://benjchristensen.com/2011/10/23/junit-tests-as-inner-classes/

[mattbishop]

How do you build the language adaptors, or where do they land?

[mattbishop]

BenI(jamin?) I am not having much luck building the language adaptors. I am using both the top-level build command as described above as well as the subproject build like this:

Projects/RxJava$ ./gradlew :language-adaptors:build

What I get in language-adaptors/build/libs is an anemic jar file (261 bytes) with only a one-line manifest. Is there another way to generate the adaptor jars?

[benjchristensen]

You can either build the entire project from the root like this:

cd RxJava
$ ./gradlew build
...
$ ls -al language-adaptors/rxjava-groovy/build/libs/rxjava-groovy-0.5.5-SNAPSHOT.jar 
-rw-r--r--  1 bechristensen  CORP\Domain Users  1115 Mar 11 13:08 build/libs/rxjava-groovy-0.5.5-SNAPSHOT.jar

Or you can go into an individual project and build it:

$ cd language-adaptors/rxjava-groovy/
$ ../../gradlew build

The contents are like this:

$ jar tvf build/libs/rxjava-groovy-0.5.5-SNAPSHOT.jar 
     0 Mon Mar 11 13:08:14 PDT 2013 META-INF/
    25 Mon Mar 11 13:08:14 PDT 2013 META-INF/MANIFEST.MF
     0 Thu Feb 28 12:09:40 PST 2013 rx/
     0 Thu Feb 28 12:09:40 PST 2013 rx/lang/
     0 Mon Mar 11 13:08:14 PDT 2013 rx/lang/groovy/
   809 Mon Mar 11 13:08:14 PDT 2013 rx/lang/groovy/GroovyAdaptor.class

[benjchristensen]

Each language-adaptor will have a build.grade file which would be changed.

For example, https://github.com/Netflix/RxJava/blob/master/language-adaptors/rxjava-groovy/build.gradle

[mattbishop]

Is scala/groovy/clojure supposed to be available or installed or such? It could be the reason I am getting such a small jar is that it gradle doesn't know what do to with these files? I built with -d to see if any error messages would surface but I couldn't spot anything obvious.

To wit: ./gradlew clean build generates this:

Projects/RxJava$ ls -la language-adaptors/build/libs/
total 24
drwxr-xr-x 5 matt staff 170 Mar 11 19:16 .
drwxr-xr-x 6 matt staff 204 Mar 11 19:16 ..
-rw-r--r-- 1 matt staff 261 Mar 11 19:16 language-adaptors-0.5.5-SNAPSHOT-javadoc.jar
-rw-r--r-- 1 matt staff 261 Mar 11 19:16 language-adaptors-0.5.5-SNAPSHOT-sources.jar
-rw-r--r-- 1 matt staff 261 Mar 11 19:16 language-adaptors-0.5.5-SNAPSHOT.jar

Which, as you can see, is a set of 261-byte jars. Each jar contains only an empty manifest file. So until I can actually build something from the git codebase, the language-adaptors can't be turned into OSGi bundles.

I'm happy to not be concerned with them in this issue as they are only used for people wanting to use them, which can be considered a separate path. Perhaps someone can raise a new issue when they want to use these languages? That would allow you to commit your change and close this issue.

[benjchristensen]

There is nothing beyond Java needed to build the project so I don't know what's going on with your build.

I have submitted a pull request with the discussed changes. Please review and let me know if they look right.

Here is the rxjava-core manifest:

Manifest-Version: 1.0
Export-Package: rx.subjects;uses:="rx.util,rx,rx.util.functions";versi
 on="0.5.6.SNAPSHOT",rx.util;uses:="rx,rx.util.functions";version="0.5
 .6.SNAPSHOT",rx.util.functions;uses:="org.slf4j";version="0.5.6.SNAPS
 HOT",rx.observables;uses:="rx.util.functions,rx";version="0.5.6.SNAPS
 HOT",rx.operators;uses:="rx,rx.util.functions,rx.util,rx.observables"
 ;version="0.5.6.SNAPSHOT",rx.plugins;version="0.5.6.SNAPSHOT",rx;uses
 :="rx.util.functions,rx.util,rx.plugins,rx.operators";version="0.5.6.
 SNAPSHOT"
Ignore-Package: junit.framework,org.mockito,org.mockito.verification,o
 rg.mockito.stubbing,org.junit
Tool: Bnd-1.50.0
Bundle-Name: rxjava-core
Created-By: 1.7.0_13 (Oracle Corporation)
Bundle-Vendor: Netflix
Bundle-Version: 0.5.6.SNAPSHOT
Bnd-LastModified: 1363058955000
Bundle-ManifestVersion: 2
Bundle-DocURL: https://github.com/Netflix/RxJava
Import-Package: org.slf4j;version="[1.7,2)"
Bundle-SymbolicName: com.netflix.rxjava.core

Here is a manifest for rxjava-groovy to show a language adaptor:

Manifest-Version: 1.0
Export-Package: rx.lang.groovy;uses:="groovy.lang,rx.util.functions";v
 ersion="0.5.6.SNAPSHOT"
Bundle-Vendor: Netflix
Bundle-Version: 0.5.6.SNAPSHOT
Tool: Bnd-1.50.0
Bundle-Name: rxjava-groovy
Bnd-LastModified: 1363058977000
Created-By: 1.7.0_13 (Oracle Corporation)
Bundle-ManifestVersion: 2
Bundle-DocURL: https://github.com/Netflix/RxJava
Bundle-SymbolicName: com.netflix.rxjava.groovy
Import-Package: groovy.lang;version="[2.1,3)",rx.util.functions;versio
 n="[0.5,1)"

Of course when released the version with be correct and not have SNAPSHOT in it.

[mattbishop]

For the language modules, please add the following instruction to the jar manifest config:

instruction 'Fragment-Host', 'com.netflix.rxjava.core'

This will give the language modules the name of the rxjava-core jar to attach themselves to, thereby allowing their classes to be joined to core's class path and thus discoverable.

Otherwise, looks great!

[benjchristensen]

Okay, I'll add that.

[benjchristensen]

What does this represent => com.netflix.rxjava.core

That is not a package or module name.

[benjchristensen]

I see that it shows up as Bundle-SymbolicName but I don't understand where that comes from as the module name is rxjava-core.

[benjchristensen]

@mattbishop Is that pull request correct?

[mattbishop]

That's the symbolic name of the rxjava-core bundle (jar). It is generated by the OSGi plugin.

Matt Bishop

On 2013-03-12, at 9:47 AM, Ben Christensen [email protected] wrote:

What does this represent => com.netflix.rxjava.core

That is not a package or module name.

—
Reply to this email directly or view it on GitHub.

[benjchristensen]

Okay ... merging now.

[benjchristensen]

This will go out in the next release.

[mattbishop]

Ben, thank you for taking the time to engage this issue so decisively even though your use has no need of OSGi. It's great to see. I'll pay attention to the issues and pitch in when OSGi comes up in the future.

On 2013-03-12, at 1:27 PM, Ben Christensen [email protected] wrote:

This will go out in the next release.

—
Reply to this email directly or view it on GitHub.

[benjchristensen]

You're welcome, I appreciate your patience (a week of travel in the middle didn't help in that regard) while I got to it.

I hope Hystrix RxJava can create value for you as it has for us.

[daveray]

@benjchristensen and by Hystrix, you meant RxJava ;)

[benjchristensen]

Ugh ... yes :-)

[benjchristensen]

That's what I get for flipping between two projects so much ... they morph into the same!

[benjchristensen]

@mattbishop Now that the code is released have you been able to confirm that it works as needed via Maven Central?

[benjchristensen]

Considering this completed as I have received no updates or complaints since releasing it.

[SimonScholz]

Hi in case you're interested I am usually using a gradle plugin to convert normal jars to osgi bundles.
See https://github.com/SimonScholz/rxjava-osgi/

I've also written a tutorial how to do that: http://www.vogella.com/tutorials/EclipseJarToPlugin/article.html#convert-jar-files-to-osgi-bundles-with-gradle