Sometimes there are JVM crashes with a SEGV, all occurring within JIT compiled code in com.adobe.cq.social.datastore.as.impl.UGCCDataServiceImpl. It is possible that there is a Java bug here, but one that seems to affect a very localized set of methods.
There is a work-around by specifying JIT parameters for this class that appears to alleviate the problem. This task is to capture and document the workaround and the circumstances where it is applicable.
- You are using Social Communities Extension with a configured Social Cloud Storage provider.
- You are running AEM on 64-bit Linux using the Oracle 64-bit JVM version 1.7.
- You are experiencing JVM crashes with Adobe Experience Manager.
- The JVM crashes are reported in com.adobe.cq.social.datastore.as.impl.UGCCDataServiceImpl
JVM crashes are accompanied by a file hs_err_pid<<PROCESS-ID>>.log appearing in the crx-quickstart directory of CQ. The basic information about the cause of the JVM crash will appear in this file. Look for something like this in the case of crashes this workaround addresses:
# A fatal error has been detected by the Java Runtime Environment: # # SIGSEGV (0xb) at pc=0x00002aaaabf4d59c, pid=17837, tid=1276991808 # # JRE version: Java(TM) SE Runtime Environment (7.0_65-b17) (build 1.7.0_65-b17) # Java VM: Java HotSpot(TM) 64-Bit Server VM (24.65-b04 mixed mode linux-amd64 compressed oops) # Problematic frame: # J 3816 C2 com.adobe.cq.social.datastore.as.impl.UGCCDataServiceImpl.readDocument(Ljava/lang/String;)Ljava/util/Map; (287 bytes) @ 0x00002aaaabf4d59c [0x00002aaaabf4d4e0+0xbc]
The critical information is the last line in this example that shows the problematic frame involves com.adobe.cq.social.datastore.as.impl.UGCCDataServiceImpl, a component of the cloud service connector.
The workaround uses two approaches: use a JVM version less prone to experience the problem, and use a JVM option to disable native compilation of the specific problem code.
At time of publishing this note, the latest Java 1.7 version available from Oracle is 1.7_65. The recommendation of this workaround is to move to 1.7_65 from any earlier version.
Using 1.7_65, the workaround is further to disable just in time (JIT) compilation of a specific problem class and method via the Java -XX:CompileCommand=exclude option. Specifically, -XX:CompileCommand=exclude,com.adobe.cq.social.datastore.as.impl.UGCCDataServiceImpl::readDocument"
This implementation will require verifying and possibly updating the version of Java you have installed and updating the startup script you use to start AEM. To upgrade Java, please see the Oracle documentation regarding choosing, downloading, installing and configuring jdk1.7.0_65 64-bit for Linux x86-64.
Generally it is recommended that AEM is started via the start scripts located in crx-quickstart/bin/start. It is common, however, for customized startup scripts to be employed. In the case where crx-quickstart/bin/start is being used, the following option must be added:
CQ_JVM_OPTS="${CQ_JVM_OPTS} -XX:CompileCommand=exclude,com.adobe.cq.social.datastore.as.impl.UGCCDataServiceImpl::readDocument"
NOTE: need to look at the default bin/start and see where this would belong, exactly.
Twitter™ and Facebook posts are not covered under the terms of Creative Commons.