Starting LiveCycle as Windows service fails with "java.lang.OutOfMemoryError: PermGen space"

After you have configured JBoss to run as a Windows service following the instructions from this link (Installing and Deploying LiveCycle ES4 for JBoss, chapter 10.2 Install the Windows service), LiveCycle will fail to start with the following exception:

java.lang.OutOfMemoryError: PermGen space at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(ClassLoader.java:792) at org.jboss.classloader.spi.base.BaseClassLoader.access$200(BaseClassLoader.java:48) at org.jboss.classloader.spi.base.BaseClassLoader$2.run(BaseClassLoader.java:637) at org.jboss.classloader.spi.base.BaseClassLoader$2.run(BaseClassLoader.java:596) at java.security.AccessController.doPrivileged(Native Method) 

If you launch JBoss from the command line, LiveCycle starts successfully.

This issue is related to an exisiting bug (# JBPAPP-5403) on JBoss EAP, as described on this link. All JVM arguments setting the memory allocation are ignored, as can be seen from the following line in the boot.log:

[ServerInfo] VM arguments: -Xrs -Djava.net.preferIPv4Stack=true -Dprogram.name=0.0.0.0 -Djava.endorsed.dirs=\jboss\lib\endorsed 

The issue can be resolved by following the steps below:

• Open "<JBoss root>\bin\service.bat" with a text editor.
• Locate the following lines:

@if not "%ECHO%" == "" echo %ECHO%
@if "%OS%" == "Windows_NT" setlocal
set DIRNAME=%CD%

• Replace with the following lines:

@if not "%ECHO%" == "" echo %ECHO%
@if "%OS%" == "Windows_NT" setlocal
set DIRNAME=%CD%

rem Read an optional configuration file.
if "x%RUN_CONF%" == "x" (
set "RUN_CONF=%DIRNAME%\run.conf.bat"
)
if exist "%RUN_CONF%" (
call "%RUN_CONF%" %*
)

• Save your "service.bat" file.
• Restart the service "JBoss for Adobe LiveCycle ES4".
• Open your "boot.log" and verify that the additional JVM arguments are present:

[ServerInfo] VM arguments: -Xrs -Dorg.jboss.resolver.warning=true -Dsun.rmi.dgc.client.gcInterval=3600000 -Dsun.rmi.dgc.server.gcInterval=3600000 -Dsun.lang.ClassLoader.allowArraySyntax=true -Xms1024m -Xmx4096m -XX:PermSize=512m -XX:MaxPermSize=768m -XX:+UseCompressedOops -Dadobeidp.serverName=server1 -Dfile.encoding=utf8 -Djava.net.preferIPv4Stack=true -DentityExpansionLimit=10000 -XX:+HeapDumpOnOutOfMemoryError -Dorg.jboss.net.protocol.file.useURI=false -Djdk.logging.allowStackWalkSearch=true -Dsun.rmi.dgc.client.gcInterval=3600000 -Dsun.rmi.dgc.server.gcInterval=3600000 -Dorg.jboss.resolver.warning=true -Djava.net.preferIPv4Stack=true -Dprogram.name=0.0.0.0 -Djava.endorsed.dirs=E:\Adobe\Adobe LiveCycle ES4\jboss\lib\endorsed

Please note, this issue only affects the Windows service created manually. The Windows service created as part of a turnkey or an automatic installation works as expected.