Sometimes there are JVM crashes with a SEGV, all occurring within JIT compiled code in com.adobe.cq.social.datastore.as.impl.UGCCDataServiceImpl. It is possible that there is a Java bug here, but one that seems to affect a very localized set of methods.
There is a work-around by specifying JIT parameters for this class that appears to alleviate the problem. This task is to capture and document the workaround and the circumstances where it is applicable.
# A fatal error has been detected by the Java Runtime Environment: # # SIGSEGV (0xb) at pc=0x00002aaaabf4d59c, pid=17837, tid=1276991808 # # JRE version: Java(TM) SE Runtime Environment (7.0_65-b17) (build 1.7.0_65-b17) # Java VM: Java HotSpot(TM) 64-Bit Server VM (24.65-b04 mixed mode linux-amd64 compressed oops) # Problematic frame: # J 3816 C2 com.adobe.cq.social.datastore.as.impl.UGCCDataServiceImpl.readDocument(Ljava/lang/String;)Ljava/util/Map; (287 bytes) @ 0x00002aaaabf4d59c [0x00002aaaabf4d4e0+0xbc]
The critical information is the last line in this example that shows the problematic frame involves com.adobe.cq.social.datastore.as.impl.UGCCDataServiceImpl, a component of the cloud service connector.
The workaround uses two approaches: use a JVM version less prone to experience the problem, and use a JVM option to disable native compilation of the specific problem code.
At time of publishing this note, the latest Java 1.7 version available from Oracle is 1.7_65. The recommendation of this workaround is to move to 1.7_65 from any earlier version.
Using 1.7_65, the workaround is further to disable just in time (JIT) compilation of a specific problem class and method via the Java -XX:CompileCommand=exclude option. Specifically, -XX:CompileCommand=exclude,com.adobe.cq.social.datastore.as.impl.UGCCDataServiceImpl::readDocument"
This implementation will require verifying and possibly updating the version of Java you have installed and updating the startup script you use to start AEM. To upgrade Java, please see the Oracle documentation regarding choosing, downloading, installing and configuring jdk1.7.0_65 64-bit for Linux x86-64.
Generally it is recommended that AEM is started via the start scripts located in crx-quickstart/bin/start. It is common, however, for customized startup scripts to be employed. In the case where crx-quickstart/bin/start is being used, the following option must be added:
NOTE: need to look at the default bin/start and see where this would belong, exactly.