Netbeans, JNI, Webstart and Apache

Although NetBeans has a Webstart module, it isn't really up to snuff yet, as I detailed in a previous post. Specifically, it doesn't support JNI code, and it can't sign the JAR files, which is needed if you have any JNI code to deploy. However, it's possible to add the necessary goop to the Netbeans build.xml file to automate this, with the help of a couple of short scripts. In addition, we can compress the JAR files using pack200 - this can give really impressive results, for example it compresses the Matisse swing-layout-1.0.2.jar file down from 155Kb down to just 19Kb! The final flourish is to get the build process to generate the Apache configuration files needed to serve Webstart files correctly. The examples below are all Win32-based as that's the environment my application works in, but they are trivially convertable to *nix versions.

The first step is to create a self-signed certificate if you don't have a proper one, the steps for doing this are described in the keytool documentation - you need to first create the certificate using the -genkey option, then sign it with the -selfcert option.

Next, we need to add some extra properties into the nbproject/ file to describe the key file and the directories we will use for the webstart build, you should edit these to reflect your setup:

# Properties needed for signing JAR files

We also need a couple of helper scripts - you should put these in the nbprojects directory:


@echo off
REM compress and sign a JAR file, remove the original
REM args are <JAR> <keystore> <password> <alias>
pack200 --repack --no-keep-file-order --strip-debug %1
jarsigner -keystore %2 -storepass %3 %1 %4
pack200 --effort=9 %1.pack.gz %1
del %1

This script will pack the supplied JAR file using the pack200 utility, and will then sign it with the supplied key. The --repack dance is necessary because packing a file with pack200 may change the order of the contents of the file and if it was signed this would break the signing, so we need to pack it, unpack it, then sign it and finally repack it again.


@echo off
REM generate a type mapping file for Apache
REM arguments are <JAR>
echo URI: packed/%1.pack.gz>%1.var
echo Content-Type: application/x-java-archive>>%1.var
echo Content-Encoding: pack200-gzip>>%1.var
echo URI: unpacked/%1>>%1.var
echo Content-Type: application/x-java-archive>>%1.var

This script generates the type mapping file needed to get Apache to serve either the normal unpacked versions or the packed versions of the files we are going to deploy. The layout of the generated file needs to be exact, down to each whitespace - if you want to understand this in detail see the pack200 deployment guide and the Apache documentation for content negotiation, but my advice is just to accept that the magic files created by the script work ;-)

Now we need to create the Apache .htaccess file and the Webstart descriptor (JNLP) file. Create a new directory - webstart_src and create the following two files in it:


AddType application/x-java-jnlp-file .jnlp
AddType application/x-java-archive .jar
AddHandler application/x-type-map .var
Options +MultiViews
<Files *.pack.gz>
    AddEncoding pack200-gzip .jar
    RemoveEncoding .gz

This assumes that your webserver httpd.conf is set up to allow the .htaccess file to function, if not see the Apache docs for how to do this. This file sets up the MIME types for the Webstart files we are going to serve, and establishes the content negotiation mechanism needed to serve the packed versions.


<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE jnlp PUBLIC "-//Sun Microsystems, Inc.//DTD JNLP 1.5//EN"
<jnlp spec="1.0+"
    <homepage href=""/>
    <description>GeocacheUK access for OziExplorer</description>
    <icon href="logo64.gif" kind="splash"/>
    <icon href="logo32.gif"/>
  <resources os="Windows">
    <j2se version="1.5+" href=""/>
    <jar href="OziGeocacheUK.jar"/>
    <jar href="OziAPI.jar"/>
    <jar href="swing-layout-1.0.1.jar"/>
    <nativelib href="OziAPINative.jar"/>
  <application-desc main-class="com.bleaklow.ozigeocacheuk.Main"/>

You should name this file appropriately for your project - in my case the Netbeans project is called OziGeocacheUK. Obviously this file will need modifying to suit your particular project - the format of JNLP files is documented here, but I'll point out the most important bits:

  • offline_allowed
    Java Web Start will also check to see if an update is available. However, if the application is already downloaded the check will timeout after a few seconds, in which case the cached application will be launched instead.
  • security/all-permissions
    The application will have full access to the client machine and local network. If an application requests full access, then all JAR files must be signed.

In my directory I also have two icon files, icon32.gif and icon64.gif which are 32x32 and 64x64 pixel icon files that I use with my application. If you do have these files, they must be GIF format, Webstart doesn't support PNG.

The final step is to add an Ant target to build a Webstart-deployable file tree to your build.xml file, the following should be fairly self-explanatory:

<!-- Create a webstart tree -->
<target name="build-webstart" depends="init,compile"
  description="Build a webstart version">
    <!-- 1. Create a fresh webstart directory -->
    <delete dir="${webstart.dist.dir}"/>
    <mkdir dir="${webstart.dist.dir}/unpacked"/>
    <mkdir dir="${webstart.dist.dir}/packed"/>

    <!-- 2. Copy the source files -->
    <copy todir="${webstart.dist.dir}">
        <fileset dir="${webstart.src.dir}"/>

    <!-- 3. Copy in all the java JAR files -->
    <copy todir="${webstart.dist.dir}/unpacked" flatten="true">
        <fileset dir="${dist.dir}" includes="**/*.jar"/>

    <!-- 4. Create a JAR file from the DLLs -->
    <jar destfile="${webstart.dist.dir}/unpacked/OziAPINative.jar"
      basedir="${dist.dir}" includes="*.dll"/>

    <!-- 5. Create packed and signed versions of the JAR files -->
    <copy todir="${webstart.dist.dir}/packed">
        <fileset dir="${webstart.dist.dir}/unpacked" includes="*.jar"/>
    <apply executable="nbproject/signpack.bat" type="file"
      resolveexecutable="true" failonerror="true" ignoremissing="false">
        <fileset dir="${webstart.dist.dir}/packed" includes="*.jar"/>
        <arg value="${keystore.location}"/>
        <arg value="${keystore.password}"/>
        <arg value="${key.alias}"/>

    <!-- 6. Sign all the unpacked JAR files -->
        storepass="${keystore.password}" >
        <fileset dir="${webstart.dist.dir}" includes="**/*.jar" />

    <!-- 7. Create the Apache type mapping files -->
    <apply executable="nbproject/genvar.bat" type="file" relative="true"
      dir="${webstart.dist.dir}" dest="${webstart.dist.dir}"
      resolveexecutable="true" failonerror="true" ignoremissing="false">
        <fileset dir="${webstart.dist.dir}/unpacked" includes="*.jar"/>
    <fixcrlf srcdir="${webstart.dist.dir}" eol="lf" eof="remove"

Running this target in Netbeans produces the following directory tree under the webstart_dist subdirectory of your Netbeans project:


This tree can then be directly deployed to an Apache webserver as it contains everything needed to serve your application. I'll cover how you can get Netbeans to automatically deploy the application in a subsequent post.

Categories : Java, Web, Tech

Building Win32 JNI code using NetBeans and MinGW

I use a GPS mapping application called OziExplorer, and just for laughs I started fiddling around with encapsulating the API for the application so that I could use it from Java. Access to the API is via a DLL, which meant delving into the scary world of both JNI and building DLLs on Win32. Whenever I've tried to do this in the past, it always seemed to require loading vast amounts of unwanted crap on my machine so that I got a compiler that worked, arcane spells involving mapfiles, sacrificial slaughter of various wildlife and so much pain in general that I'd always given up in disgust. However, I'm a glutton for punishment so I thought I'd have another crack at it. I couldn't find anything on the web that told you everything you needed to know all in one place, so I thought I'd document it since I'd got it all working.

Step 1

If you don't already have it, download and install the latest version of the Java Development Kit. I used version 1.5.0_07, the instructions below should work with other versions... Do this first to make you life easy - the NetBeans installer will pick up the latest version of the JDK that you have installed.

Step 2

If you don't already have it, download Netbeans and install it. I used version 5.0, that or anything later should be fine.

Step 3

Install the Ant cpptasks module. This adds support to Ant for building C code. The only bit of this you need is cpptasks.jar, and you should copy it to the lib subdirectory of the new version of Ant you installed in step 3.

Step 4

Install a copy of MinGW. This is a Win32 version of the well-know gcc compiler. Unfortunately the MinGW documentation is as clear as mud on exactly how to do this - if you go to the download page you get a huge shopping list of bits, with no clear directions as to which bits you actually need - the documentation is clearly written with the assumption that you already know. The easiest way to sort the mess out is to download the "MinGW" installer - look for the section entitled "MinGW" and you'll see a file called MinGW-<version>.exe, in my case <version> was 4.1.0. Download and run that and it will ask you for an installation location (I used C:\Program Files\MinGW, then deselect everything except The minimal set of packages required to build C/C++. Leave everything on the next screen selected and start the install. The installer will then pull the bits you need from the web. You can probably prune the list, but the whole thing is only 60Mb so it didn't seem worth the effort.

Step 5

Set up your environment. In order for Ant to be able to run the compiler, it will need to be able to find it, and the same goes for any DLLs that you will use or generate. Windows particularly sucks in this area, so to save spraying DLLs all over my C drive I created the directory C:\DLLs to dump everything in. With that done, right click on My Computer, click on Advanced then Environment Variables. Select Path from the list, then add ;C:\Program Files\MinGW\bin;c:\DLLs to the end, or wherever you installed the compiler, and wherever you intend to install your DLLs. If it isn't already there, add the path to the Java bin directory as well - in my case this is ;C:\Program Files\Java\jdk1.5.0_04\bin. I don't know if it is necessary, but at this point I logged out/into Windows to make sure my environment changes took effect.

Step 6

Add the appropriate JNI code to your project - I don't intend to cover the process for doing that here, instead see the JNI documentation. The tricky bits are getting everything to link together, the problem being caused by the awful Win32 name decoration mess. Basically, DLLs usually use the stdcall calling convention, and when you build them with gcc by default it 'decorates' the symbol names by appending '@' and a number (the number of bytes that will be popped off the stack by the function) - follow the links for details. There are two main issues:

  • In the case of the DLL I was wrapping, the functions used the stdcall convention but didn't have the name decorations expected by gcc. The solution was to use a def file in concert with dlltool to generate a gcc library file - then instead of linking directly with the DLL, you link against the library file. The def file itself is just a list of the decorated names that gcc is expecting to see in the DLL - you can get these by looking at the linker output, when gcc can't find the decorated names when it is linking it helpfully prints out the missing symbol names. Beware - if you do build without using stdcall your code will sort-of work, but because the stack isn't being cleaned up properly you'll get all sorts of odd crashes.
  • In the case of the DLL I was generating (containing the JNI code), by default gcc adds the name decoration to the symbols but the Java runtime doesn't look for the decorated symbols, so you get run-time linker errors from Java when you try to load the DLL. The solution here is quite simple - add the --kill-at flag to the gcc command line and gcc will generate your DLL without the name decorations, and Java will be happy.

Step 7

Javap can be used to output the method and field signatures that will be needed in the JNI C code. Unfortunately there isn't an Ant target to do this, so it's necessary to roll our own. The first thing is to put the following 2-line Windows batch script somewhere - I called mine javap-s.bat and put it under the nbproject subdirectory of my NetBeans project. See the following section for the explanation of how to hook it into Ant.

@echo off
javap -s -private -classpath %~dps1 %~n1 > %~s2

Step 8

The next tricky bit is adding the necessary goop to your project.xml file to get Ant to build your code bearing in mind the issues I described above. This isn't helped by the fact that the cpptasks add-on has a bug that means it doesn't work properly with MinGW. Here's what works for me. Firstly, add the following to the nbproject/ file:


Then add the following to the build.xml file:

    <!-- Load the cpptasks task -->
    <taskdef resource="cpptasks.tasks"/>
    <typedef resource="cpptasks.types"/>

    <!-- Compile the JNI code into a DLL -->
    <target name="-post-compile">
        <!-- Make sure the output directories exists -->
        <mkdir dir="${jni.dir}"/>
        <mkdir dir="${signatures.dir}"/>

        <!-- Run javah to produce a header file for the JNI functions -->
        <javah verbose="yes" classpath="${build.classes.dir}"
            <class name="com.oziexplorer.OziAPI"/>

        <!-- Run javap to produce files containing the JNI signatures -->
        <apply executable="nbproject/javap-s.bat" dest="${signatures.dir}"
          resolveexecutable="true" failonerror="true" ignoremissing="false">
            <fileset dir="${build.classes.dir}/com/oziexplorer"
            <mapper type="glob" from="*.class" to="*.txt"/>

        <!-- Check the library definition file is up to date -->
        <apply executable="dlltool" dest="${jni.dir}" failonerror="true"
            <filelist dir="${src.dir}/com/oziexplorer" files="OziAPI.def"/>
            <mapper type="glob" from="*.def" to="lib*.a"/>
            <arg value="-k"/>
            <arg value="--dllname"/>
            <arg value="OziAPI.dll"/>
            <arg value="--def"/>
            <arg value="--output-lib"/>

        <!-- Compile the C code -->
        <cc link="shared" outtype="shared" multithreaded="true" optimize="speed"
          objdir="${jni.dir}" outfile="${jni.dir}/OziAPI">
            <compilerarg value="-Wall"/>
            <compilerarg value="-D_JNI_IMPLEMENTATION_"/>
            <compilerarg value="-fno-strict-aliasing"/>
            <linker name="gcc">
                <linkerarg value="--kill-at"/>
                <linkerarg value="-oOziAPIJava.dll"/>
            <sysincludepath location="${java.home}/../include"/>
            <sysincludepath location="${java.home}/../include/win32"/>
            <fileset dir="${src.dir}/com/oziexplorer" includes="OziAPI.c"/>
            <libset dir="${jni.dir}" libs="OziAPI"/>

    <!-- Copy the stripped DLL and OziAPI.DLL to the dist directory -->
    <target name="-post-jar">
        <apply executable="strip" dest="${dist.dir}" failonerror="true"
            <filelist dir="${jni.dir}" files="OziAPIJava.dll"/>
            <mapper type="glob" from="*.dll" to="*.dll"/>
            <arg value="-s"/>
            <arg value="-o"/>
        <copy todir="${dist.dir}">
            <fileset dir="${lib.dir}" includes="*.dll"/>

Let's go through it bit at a time:

  • The typedef statements pull in the cpptasks extension.
  • The mkdir makes sure the output directory exists - this rule runs before the JAR file is generated, so we need to manually create the output directory.
  • The javah section runs javah to generate a C header file from the file, which contains the native function definitions. This is useful to make sure you get the function prototypes correct.
  • The next section runs the batch file created in step 8 over each of the generated class files to produce the corresponding text files containing the signatures of the methods and fields. These are placed in the source directory, but that can be changed to suit.
  • There also isn't any obvious way of running dlltool from within the cpptasks framework, so it's necessay to manually check the dependencies between the def file and the resulting library, and rebuild if necessary. That's done by the second apply section.
  • The cc section has a few things that need clarification:
    • The linkerarg with the -o is to work around the bug I mentioned above - without it the cc task thinks that it is running on Unix rather than Win32 and generates instead of OziAPIJava.dll.
    • --kill-at stops the linker putting the @ decoration on the generated code and means that the functions can be linked to correctly.
  • The -post-jar section strips and copies the resulting DLL to the directory established in step 6, along with any other required DLLs that have been put in the ${lib.dir} directory.

Although it took a bit of fiddling to get all the bits to play together, the process of building DLLs is far easier than it was in the past - the work that the gcc and MinGW crowd have done to do all the dirty work for you is really very impressive.

Categories : Java, Tech

Using NetBeans with SourceForge (Win32)

I have a Win32 project hosted on SourceForge, and as it took me a bit of fiddling to get NetBeans to work with the SourceForge CVS server I thought I'd document the steps needed - although there is some information on the NetBeans and SourceForge sites, there isn't an all-in-one guide.

The prerequisites are that you already have NetBeans installed, and that you have an account on SourceForge.

The first step is to grab a copy of PuTTY if you don't already have it - PuTTY is a telnet/ssh client and you'll need it to connect via SSH to SourceForge.

Once you have PuTTY installed, use the PuTTYgen utility to generate a new SSH key. It's easiest if you save the keys without a passphrase - if you don't you'll have to run the Pageant authentication agent that comes with PuTTY and load your key into it before trying to connect to SourceForge. If you do decide to save them without a passphrase, make sure you put the files somewhere safe! Save both the public and private key files - I used the filenames and SourceForge-shell.ppk. Also export the private key as a OpenSSH key (Conversions->Export OpenSSH Key), I saved mine in a file called SourceForge-shell.openssh. Leave PuTTYgen open - you'll need it in the next step.

Go to the Edit SSH Keys page on SourceForge and cut+paste the contents of the box in PuTTYgen labelled Public key for pasting into OpenSSH authorized_keys file into the SourceForge Authorized keys: form, and press submit. It will take at least 10 minutes for SourceForge to update your keys.

After waiting, check you can connect to the CVS server on SourceForge from the command-line before setting up NetBeans. Start up a command prompt and run the PuTTY plink command to check that you can connect to the SourceForge CVS server. In the example below, substitute in the appropriate values for your setup for all the my_ strings.

$ c:\progra~1\putty\plink -ssh -i c:\my_safe_dir\SourceForge-shell.ppk
Using username "my_sf_id".
Last login: Wed May 27 16:20:41 2006 from

Welcome to

This is a restricted Shell Account
You cannot execute anything here.


Once you have checked you can access SourceForge CVS from the command-line, set up access in NetBeans. Select CVS->Checkout and for CVS Root enter (substituting the my_ values appropriately, as in the command-line test) Select the Use External Shell button and in the SSH Command box enter c:\program files\putty\plink -ssh -i c:\my_safe_dir\SourceForge-shell.ppk, then press Next. NetBeans will then connect to SourceForge and you can browse and checkout modules from CVS.

One thing to note - if you subsequently come back to the Checkout dialog, the Use External Shell setting doesn't 'stick', so you will have to relselect the button before pressing Next.

Categories : Java, Tech

Fixing O'Reilly Safari Books Online annoyances, part 2

As I said in my last post, I like O'Reilly's Safari Online, but I find some of the presentation a little annoying. Other than the grotty font choices, my other bugbear is the awful "Additional reading" section that appears at the bottom of lots of the pages. The blurb says:

Safari has identified sections in other books that relate directly to this selection using Self-Organizing Maps (SOM), a type of neural network algorithm. SOM enables us to deliver related sections with higher quality results than traditional query-based approaches allow.

but high-quality it isn't, for example whilst viewing a page about SQL queries I get a suggestion that I go read some chapter in a C# book! Whilst I could fix the font problem with Firefox's userContent.css file, I couldn't use that method to excise the "Additional reading" cruft, as although it was in a div the div in question had neither a class nor an id. This looked like a job for Greasemonkey. Greasemonkey is a Firefox plugin that allows you to run user-defined Javascript over each page as it is loaded by the browser, so you can modify page content before it is displayed. Earlier versions of Greasemonkey had some serious security flaws, but these have been fixed in the current version. With Greasemonkey installed, removing the cruft was a snap - each section I wanted to chop out was enclosed in a div as I said, and had a h4 heading containing the string Additional reading as its content. As I was going to have to use Greasemonkey, I reimplemented the CSS hack I described in my last post in the Greasemonkey script - the advantage being that individual Greasemonkey scripts can be enabled and disabled, unlike userContent.css, which is permanent. The following script did the trick:

// ==UserScript==
// @name		Safari Books Online cleanup
// @namespace
// @description		Fix fonts and remove 'Additional reading' section from Safari pages
// @include		http://**
// ==/UserScript==

/* Make the fonts readable. */
GM_addStyle(' \
	.docText, .docList { \
		font-family:	sans-serif	!important; \
		font-size:	medium		!important; \
	} \
	.docFootnote, .docItemizedlist { \
		font-family:	sans-serif	!important; \
		font-size:	small		!important; \
	} \
	tt, pre, code, .docMonoFont { \
		font-family:	monospace	!important \
	} \
} \

/* Remove cruft from the top of the page. */
var node;
var nodes = document.evaluate(
    document, null, XPathResult.ORDERED_NODE_SNAPSHOT_TYPE, null);
if (nodes.snapshotLength == 2) {
	node = nodes.snapshotItem(0).parentNode;
	node = nodes.snapshotItem(1);

/* Remove the stupid "Additional reading" section. */
node = document.evaluate(
    '//h4[.="Additional reading"]/..',
    document, null, XPathResult.ANY_UNORDERED_NODE_TYPE, null).singleNodeValue;
if (node != null) {

/* Remove the book cover and details and replace with a simple heading. */
node = document.evaluate(
    '(//a[@title="Book Cover"])[1]/../..',
    document, null, XPathResult.ANY_UNORDERED_NODE_TYPE, null).singleNodeValue;
if (node != null) {
	var td_cover = node.childNodes[0];
	var tbody_info = node.childNodes[1].childNodes[0].childNodes[0];
	var title =
	while (tbody_info.childNodes.length > 1) {
	tbody_info.childNodes[0].innerHTML = "<h2>" + title + "</h2>";

The important bits to note are the @include, which restricts this script to just the Safari website, and the use of document.evaluate with an XPath expression to find the appropriate part of the DOM to tweak.

Tags : ,
Categories : Web, Tech

Fixing O'Reilly Safari Books Online annoyances

While I love the content available from O'Reilly's Safari Books Online website, I'm less than enamoured with the presentation. Specifically it insists on using small Times Roman as the font for displaying the contents of books. You can increase the size, but it doesn't 'stick' between visits, and my aging eyesight really doesn't cope well with small serif fonts, especially the rather grotty Times Roman I have available - rather than specifying a serif font, the CSS explicitly asks for Times Roman.

The fix is pretty straighforward - firstly I found the CSS style used by the text in question, using the most excellent Web Developer extension for Firefox - install the extension, press <Control-Shift-Y> and click on the offending text to find the style information.

Next step: override the CSS by using Firefox's userContent.css file to provide a replacement. This file is found in your Firefox preferences directory, the easiest way to locate it is with find $HOME/.mozilla -name userContent-example.css - it should live in the same directory as that file. Here's what I used:

@-moz-document domain( {
        .docText, .docList {
                font-family:    sans-serif      !important;
                font-size:      medium          !important;
	.docFootnote, .docItemizedlist {
		font-family:	sans-serif	!important;
		font-size:	small		!important;

Note that after you've created this file you need to restart Firefox, as it only gets read when Firefox starts up. The neat bit is the use of the @-moz-document directive to restrict the custom CSS to just the Safari Books Online website, as .docText and .docList aren't exactly unusual names and could conveivably used by other websites. The full synxax of the @-moz-document directive is as follows:

  • url(
    Apply the CSS to the page

  • url-prefix(
    Apply the CSS to any page whose URL begins with

  • domain(
    Apply the CSS to any page whose URL's host is or ends with

Tags : ,
Categories : Web, Tech