Getting the GlassFish 3.1.2 updatetool to work on Ubuntu/Mint 64-bit

I wrestled with the GlassFish updatetool monster for a couple of hours, and I decided to spare someone else the trouble of going through this all over again.

If you're using GlassFish 3.1.2 on a 64-bit OS, you're likely to run into a couple of bugs related to the GlassFish update tool. More likely so, if you've installed GlassFish off the ZIP bundles available at glassfish.java.net - extracting these archives do not install the updatetool utility.

The JVM crash

I ran into a segfault when running the updatetool from the commandline. Portions of the reported fault are listed below:


Proxy: Using system proxy settings.
Install image: /opt/glassfish-3.1.2.2/glassfish3
Installing pkg packages.
Downloading 2 packages.
Downloading pkg (511 files, 6,237,937 bytes).
thread 140690402031360 also had an error]
#
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0x00007ff50b573920, pid=25590, tid=140690403084032
#
# JRE version: 7.0_09-b05
# Java VM: Java HotSpot(TM) 64-Bit Server VM (23.5-b02 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# C [libdbus-1.so.3+0x26920] dbus_watch_handle+0x1b20
#
# Core dump written. Default location: /opt/glassfish-3.1.2.2/glassfish3/bin/core or core.25590
#
# An error report file with more information is saved as:
# /opt/glassfish-3.1.2.2/glassfish3/bin/hs_err_pid25590.log


Opening the pid error log showed the actual cause:


Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
j sun.net.spi.DefaultProxySelector.getSystemProxy(Ljava/lang/String;Ljava/lang/String;)Ljava/net/Proxy;+0
j sun.net.spi.DefaultProxySelector.access$100(Lsun/net/spi/DefaultProxySelector;Ljava/lang/String;Ljava/lang/String;)Ljava/net/Proxy;+3
j sun.net.spi.DefaultProxySelector$2.run()Ljava/net/Proxy;+151
j sun.net.spi.DefaultProxySelector$2.run()Ljava/lang/Object;+1
v ~StubRoutines::call_stub
j java.security.AccessController.doPrivileged(Ljava/security/PrivilegedAction;)Ljava/lang/Object;+0
j sun.net.spi.DefaultProxySelector.select(Ljava/net/URI;)Ljava/util/List;+223
j sun.net.www.protocol.http.HttpURLConnection.plainConnect()V+314
j sun.net.www.protocol.http.HttpURLConnection.connect()V+1
j sun.net.www.protocol.http.HttpURLConnection.getInputStream()Ljava/io/InputStream;+195
j java.net.HttpURLConnection.getResponseCode()I+16
j com.sun.pkg.client.Image.checkRepositoryConnection(Ljava/net/HttpURLConnection;)V+1
j com.sun.pkg.client.DownloadFileTask.download()Ljava/io/File;+73
j com.sun.pkg.client.DownloadFileTask.call()Ljava/io/File;+1
j com.sun.pkg.client.DownloadFileTask.call()Ljava/lang/Object;+1
j java.util.concurrent.FutureTask$Sync.innerRun()V+29
j java.util.concurrent.FutureTask.run()V+4
j java.util.concurrent.ThreadPoolExecutor.runWorker(Ljava/util/concurrent/ThreadPoolExecutor$Worker;)V+46
j java.util.concurrent.ThreadPoolExecutor$Worker.run()V+5
j java.lang.Thread.run()V+11
v ~StubRoutines::call_stub


This turned out to be GLASSFISH-18360. Applying the workaround listed in that report was sufficient to bypass the problem - you'll merely need to update the updatetool script to set the value of "proxy.use.system" to false. Of course, this works if you're not using a proxy.

Now, you should be able to install the updatetool utility.

The 32-bit libraries

That wasn't good enough though. Running updatetool after installation, resulted in this message:


Locale could not be determined. Attempting to use English locale.
WX import error. Verify the WX widgets are in the PYTHONPATH.
The following can be reported to GlassFish Update Tool 2.3.5 Development Team <dev@updatecenter.java.net>.

Traceback (innermost last):
File "/opt/glassfish-3.1.2.2/glassfish3/updatetool/vendor-packages/updatetool/common/boot.py", line 283, in init_app_locale
import wx
File "wx/__init__.py", line 45, in ?
File "wx/_core.py", line 4, in ?
ImportError: libjpeg.so.62: cannot open shared object file: No such file or directory

---------------------------------------------------------------
There was an error running

/opt/glassfish-3.1.2.2/glassfish3/updatetool/bin/../../pkg/python2.4-minimal/bin/python


The weird bit about this was that the ia32-libs package required by updatetool was already installed. The problem turned out to be the use of the 64-bit version of libjpeg:

$ locate libjpeg.so
/usr/lib/jvm/jdk1.7.0_07/jre/lib/amd64/libjpeg.so
/usr/lib/x86_64-linux-gnu/libjpeg.so.8
/usr/lib/x86_64-linux-gnu/libjpeg.so.8.0.2

On 64-bit Mint or Ubuntu, you'll need to explicitly install the i386 version:

sudo apt-get install libjpeg62:i386

That should do be sufficient:

$ locate libjpeg.so
/usr/lib/i386-linux-gnu/libjpeg.so.8
/usr/lib/i386-linux-gnu/libjpeg.so.8.0.2
/usr/lib/jvm/jdk1.7.0_07/jre/lib/amd64/libjpeg.so
/usr/lib/x86_64-linux-gnu/libjpeg.so.8
/usr/lib/x86_64-linux-gnu/libjpeg.so.8.0.2

Running updatetool now should bring up the GUI.

The embedded EJB container in WebLogic Server 12c

The EJB 3.1 specification requires that all containers support embedded usage, with the additional requirement to support the EJB Lite feature set in an embedded container environment.

WebLogic 12c has had this feature since it's first release in Dec 2011, but apart from some sparse documentation on OTN, there was no material describing it's usage. I could not get it to work in a suitable manner, when I first attempted to write an Arquillian adapter earlier this year. Since then, I've had time to "peek" into it's internals and get it working.

Before we proceed, I'd like to issue a warning - use this at your own risk. Some of this is not documented in the WebLogic Server documentation at this time, and some use cases are most likely not supported by Oracle. These notes are based on the WLS 12c developer installation pack hosted on OTN.

The embedded EJB container provider

Starting the embedded container

The WLS embedded EJB container is initialized just like other embeddable containers:

EJBContainer container = javax.ejb.embeddable.EJBContainer.createEJBContainer();

At this point, an implementation of the javax.ejb.spi.EJBContainerProvider interface is located and loaded from the classpath. In the case of WLS 12c, the implementation and it's dependent classes can be found in weblogic.jar. It is also possible to use wlclient.jar or wlfullclient.jar instead, but I ended up running into CNFEs that I did not want to pursue; documentation of the required JARs for the embedded container would have helped here.

Note - copying over weblogic.jar and using it in your project is also bound to result in CNFEs. The weblogic.jar file uses relative paths in it's ClassPath manifest entry, and hence you would be required to replicate the entire file system structure of WL_HOME/modules and WL_HOME/server/lib at a minimum.

Note about using Surefire - If you're using Surefire to run your tests that depend on the EJBContainer API, you should add weblogic.jar as an additionalClasspathElement entry in your POM. Of course, you'll need to use an absolute path, but this way, you avoid the problems associated with installing weblogic.jar as a Maven artifact in your repository - you no longer need to figure out how to create a uber weblogic.jar or manage the transitive dependencies of the same.

Using the embedded container in a test

The most common use of an embedded container is in running tests. While embedded containers have their disadvantages over their standalone counterparts, developers tend to prefer embedded containers primarily for their startup performance.

For the sake of demonstrating a test involving an embedded container, consider a SLSB with a no-interface view:

package com.acme.ejb;

import javax.ejb.LocalBean;
import javax.ejb.Stateless;

@Stateless
@LocalBean
public class Greeter {

    public String sayHello(String name) {
        return "Hello " + name;
    }

}

The following JUnit4 test demonstrates the usage of the embedded container to test the SLSB:

package com.acme.test; 

import static org.hamcrest.core.IsEqual.equalTo; 
import static org.hamcrest.core.IsNull.notNullValue; 
import static org.junit.Assert.assertThat; 

import javax.ejb.embeddable.EJBContainer; 
import javax.enterprise.inject.spi.BeanManager; 
import javax.naming.Context; 

import org.junit.AfterClass; 
import org.junit.Before; 
import org.junit.BeforeClass; 
import org.junit.Test; 

import com.acme.ejb.Greeter; 

public class GreetingTest { 

    private static Greeter greeter; 
    private static EJBContainer ejbContainer; 
    private static Context ctx; 

    @BeforeClass 
    public static void setupContainer() throws Exception { 
        ejbContainer = EJBContainer.createEJBContainer(); 
        ctx = ejbContainer.getContext(); 
    } 

    @Before 
    public void setup() throws Exception { 
        greeter = (Greeter) ctx.lookup("java:global/classes/GreeterBean"); 
    } 

    @AfterClass 
    public static void shutdownContainer() throws Exception { 
        if (ejbContainer != null) { 
            ejbContainer.close(); 
        } 
    } 

    @Test 
    public void testGreeter() throws Exception { 
        String message = greeter.sayHello("World"); 
        assertThat(message, equalTo("Hello World")); 
    } 

}

While the @Test method itself completes in the order of milliseconds, the rest of the setup activity involving startup of the embedded container and deployment of the EJB runs into a few seconds (~20 on my workstation). If you are testing components of your application that use the EJB Lite features, this is a possible way to cut down on test execution time. I am of course assuming that you do not run into the typical issues involving embedded container usage.

I found it a bit surprising when I had to use the java:global namespace to lookup the deployed SLSB. Although the container bound the SLSB to several JNDI names, including the java:app and java:module namespaces, using the other names during lookup failed. This is either a bug, or a feature - WLS allows lookups in these namespaces from within the SLSB but not from the test class.


Note - the embedded container of WLS does not truly shutdown when the EJBContainer.close() method is invoked. A few user (non-daemon) threads continue to run, and therefore a complete shutdown occurs only when System.exit/Runtime.exit is invoked. Failure to invoke System.exit/Runtime.exit would leave the JVM in a running state. TestRunners usually do not have this problem, but if you're starting the embedded container in a main() method, you would need to terminate the JVM explicitly.

Permgen space issues

The embedded EJB container of WLS is not as lightweight as I expected it to be. It runs into issues with the space allocated to the permanent generation consistently. I used the below listed JVM flag to boot up the JVM that eventually runs the embedded container, to make the problem go away.

-XX:MaxPermSize=128M

The JVM had a max heap utilization of ~300 MB, and a max perm gen size of ~100 MB for the deployment of a single SLSB. Apparently, other services were also started along with the EJB container, and I didn't find a way to prevent them from starting (maybe they are necessary).

You may need to increase the heap-size as well as the permanent generation size if your application is larger.

Using an existing WLS domain

Note - this appears to be an undocumented feature at the moment. Use it at your own risk.

The embedded EJB container creates a new WebLogic domain upon initialization; the domain is created in the "java.io.tmpdir" directory. Sometimes, you might want to use an existing WLS domain, especially if you'd like to cut down on startup time. This is possible, but the mechanism is very kludgy and requires some knowledge of the WLS container startup mechanisms.

Unlike the GlassFish embedded container that supports usage of an existing GlassFish installation and instance, through the setInstallRoot() and setInstanceRoot() methods of the BootStrapProperties class, embedded WLS does not appear to have any documented/supported mechanism at the moment. However, a peek into the embedded container reveals a few things. The embedded EJB container looks for an existing WLS domain at the location defined by the "weblogic.RootDirectory" system property. Using this property one could point the embedded container to use a previously created WLS 12c domain, like so:

System.setProperty("weblogic.RootDirectory", "/home/vineet/Oracle/Middleware/user_projects/domains/embedded_domain");

This is still not sufficient though. The embedded container looks for a marker file ".embed-server-marker" at the root of the domain directory. An absence of the marker file will result in the container throwing an exception on startup. You can create a marker file in Linux, like so:

touch ~/Oracle/Middleware/user_projects/domains/embedded_domain/.embed-server-marker


Again, this is not sufficient to operate the embedded container with an existing domain. The boot identity would have to be passed to the container at startup. This is done via the "weblogic.management.username" and "weblogic.management.password" properties, like so:

System.setProperty("weblogic.management.username","weblogic");
System.setProperty("weblogic.management.password","welcome1"); 

Finally, (no, it's not over yet) you'll need to configure the domain to use a special startup class. In your $DOMAIN_HOME/config/config.xml file, add the following startup class that targets the admin server of your domain:

  <startup-class>
    <name>embedded-server-startup-class</name>
    <target>myserver</target>
    <class-name>weblogic.server.embed.internal.EmbeddedServerStartupClass</class-name>
    <failure-is-fatal>true</failure-is-fatal>
    <load-after-apps-running>true</load-after-apps-running>
  </startup-class> 

It is important to note that the Admin Server of the WLS domain is the one that is started in the embedded container.

This is also the time to reflect on why the marker file created earlier is important - domains created by the embedded container are configured in a special way (with the startup class at a minimum), and are possibly not recommended for continuous development. My assumption is that using the embedded container against an existing domain might change the domain configuration in an irrevocable manner, and hence the absence of marker file protects a typical WLS domain from these changes.

Starting WebLogic Server (10.3/11gR1) in debug mode

This is a mere note.
When starting WebLogic Server using the startWebLogic script, the JDWP options for starting the server can be enabled, by setting the 'debugFlag' environment variable to true.

The startWebLogic script examines the value of this environment variable to add the remote debugging flags to the JVM that it initializes.