2008-12-04

Bundle-NativeCode in the OSGi manifest

OSGi is a pretty straight forward and quite easy to use technology. As long as you are doing the thing Java programming language. But, what happens when you start to involve other implementations, i.e. native code (most often compiled C/C++ source). On Windows these libraries are known as Dynamic Link Libraries, or DLL for short. Linux and Unix clones are called shared object libraries and has the .so suffix.
What about Mac OS X? Well, there is also so-files, but I've also encountered so called dynlib:s. Anyhow, do not despair, all these are nicely handled by OSGi. I.e. an OSGi framework will happily load these libraries into the JVM process space and thus make them available to whatever Java class that is about to call them. If you are about to produce some native code, check out the tutorial at Sun.

Ok, you have your Java class defining the native methods, generated the .h-file from it and done the mandatory HelloWorld thing on the native side, also got the System.loadLibrary call right and everything. It works all nice and peachy.

So, now what? How to get it to work encapsulated within an OSGi bundle? as you've got all teh basic stuff in place, it is pretty much a walk in the park. Well, all you actually need is to declare your newly created native library in the bundle manifest and also declare which library goes with which operating system. Yes, you can bundle several libraries for various operating systems in the same bundle and the framework load the correct one depending on the declaration in the manifest.

So how does this portion of the manifest look like? Here is a simple example of declaring a DLL (Windows) named "mylib", located in a directory named "lib":

.
.
Bundle-NativeCode: lib/mylib.dll ;
osname=Win32 ;
processor=x86
.
.

Please note the placement of the semicolons (';'). That is quite important.

But; what if I've got several DLL:s, or libraries, in my project that need to go into the same bundle? Well, that is not a problem. What you need to keep in mind is the placements of the semicolons.

So, expanding the above simple example with two DLL:s (mylib1 and mylib2) for the same platform it will look like this:

.
.
Bundle-NativeCode: lib/mylib1.dll ; lib/mylib2.dll ;
osname=Win32 ;
processor=x86
.
.

Ok, everything is nice and peachy. Next iteration of the projects adds a whole bunch of requirements, one is regarding the support for yet another platform: Linux. Can I easily add that?
Sure! Produce the necessary native libraries for the new platform, add them into your project and finally - before launching the debugger - add some more lines to the manifest:

.
.
Bundle-NativeCode: lib/mylib1.dll ; lib/mylib2.dll ;
osname=Win32 ;
processor=x86,
lib/libmylib1.so ; lib/libmylib2.so ;
osname=linux;
processor=x86
.
.

See the comma (',')? Each "block" is separated by a comma. As you also might have guessed by now is that fact that not only is it possible to qualify on operating system. You can also load different libraries depending on processor architecture, on the same operating system, and operating system version. Further, I haven't discussed another option that is quite powerful; the filter capability. It is also possible to qualify on basically anything, e.g. which windowing system is present and thus load an appropriate library dependent on that.

Maybe I'll discuss how to make use of the filter capability in a future posting. Anyhow, I do recommend that you download and read the OSGi specification from osgi.org, it is one of the best specifications when it comes to readability I've come across. The section on Bundle-NativeCode is very worth studying, it discusses the above and more ...

4 comments:

Michael Grammling said...

Hi,
indeed OSGi could really help to manage native code in a better way (also with the update option).
Anyway it should me mentioned that you have to develop native code directly for OSGi.
For example if you would like to integrate legacy native code or third party code consisting of more than one DLL or SO file you will run into trouble.
The problem occurs if the native files have dependencies to each other. Generally a DLL finds its dependent DLLs by looking in the library path.
In OSGi you would have to load all DLLs or SO files manually by using System.loadLibrary(...). This means also that you have to find out the dependencies to all DLLs or SO files.
I run into heavy trouble under Linux because it seems that the JVM loads native libraries locally instead of globally which means that the manual loading mechanism with System.loadLibrary(...) does not work anymore.
The bundle update process causes much more problems then because the framework implementations rename the native libraries because the JVM does not offer a direct unload library command.

Fazit: OSGi offers a great possibility by managing native code as long as you're only working with JNI code which has no dependencies to other DLLs.
// michael

R. Varttinen said...

Great comment Michael! Thank you!
Yes, one has to fiddle around with the LD_LIBRARY_PATH on Linux/Unix systems, PATH on Win32. And, yes it is a bit cumbersome at times (well, most of the time actually).
Some people I know use the technique you mention, calling System.loadLibrary(..) for each library needed in the process space.
There are some dependency walkers available to help in this respect.
Another way I've seen; statically link those libraries yo need, but that can make some libraries rather large. Nothing I'll recommend for anything but quite small things...

To sum up: doing stuff on the native side is often tricky, we are outside the Java sandbox and that has to be taken into consideration when doing these kind of things. It is "unsafe" in many respects ...

Ralf said...

:)

Vinay said...

Hi,
Thank you for this wonderful article. It is very informative.

I have a scenario where my Dll is depending on an external library. In such a case my bundle start fails saying cant find dependent libraries. Please let me know how to handle such dependencies.

Thank you,
Vinay