AEM crashes with OutOfMemoryError after PDF upload caused by Oracle JVM

ค้นหา

อัปเดตครั้งล่าสุดเมื่อ 21 ธ.ค. 2021

Issue

After uploading certain PDFs, the AEM instance crashes. In the log files, you see OutOfMemoryError. In thread dumps, you see threads like the one below showing threads stuck in the JVM code sun.java2d.cmm.lcms.LCMS.createNativeTransform:

"JobHandler: /etc/workflow/instances/server0/2017-02-16/update_asset_833:/content/dam/geometrixx/documents/Test.pdf/jcr:content/renditions/original" #168 daemon prio=1 os_prio=0 tid=0x00007ff525c59800 nid=0x1ab7 runnable [0x00007ff3feffc000]
   java.lang.Thread.State: RUNNABLE
    at sun.java2d.cmm.lcms.LCMS.createNativeTransform(Native Method)
    at sun.java2d.cmm.lcms.LCMS.createTransform(LCMS.java:156)
    at sun.java2d.cmm.lcms.LCMSTransform.doTransform(LCMSTransform.java:155)
    - locked <0x000000077e9ae718> (a sun.java2d.cmm.lcms.LCMSTransform)
    at sun.java2d.cmm.lcms.LCMSTransform.colorConvert(LCMSTransform.java:268)
    at java.awt.image.ColorConvertOp.ICCBIFilter(ColorConvertOp.java:355)
    at java.awt.image.ColorConvertOp.filter(ColorConvertOp.java:282)
    at com.adobe.internal.pdftoolkit.image.ARGBImage.convertToRGB(ARGBImage.java:340)
    at com.adobe.internal.pdftoolkit.image.ARGBImage.handleICCColorSpace(ARGBImage.java:177)
    at com.adobe.internal.pdftoolkit.image.ARGBImage.convertToRGB(ARGBImage.java:310)
    at com.adobe.internal.pdftoolkit.image.ARGBImage.access$1300(ARGBImage.java:59)
    at com.adobe.internal.pdftoolkit.image.ARGBImage$ARGBTilesIterator.ConvertToRGB(ARGBImage.java:587)
    at com.adobe.internal.pdftoolkit.image.ARGBImage$ARGBTilesIterator.hasNext(ARGBImage.java:640)
    at com.adobe.internal.pdftoolkit.services.rasterizer.impl.RasterContentImageItem.writeToDisplayArea(RasterContentImageItem.java:211)
    at com.adobe.internal.pdftoolkit.services.rasterizer.impl.RasterDocument.createPage(RasterDocument.java:131)
    at com.adobe.internal.pdftoolkit.services.rasterizer.impl.PDFToRasterConverter.toBufferedImage(PDFToRasterConverter.java:127)
    at com.adobe.internal.pdftoolkit.services.rasterizer.PageRasterizer.next(PageRasterizer.java:98)
    at com.day.cq.dam.handler.standard.pdf.PdfHandler.getImage(PdfHandler.java:583)
    at com.day.cq.dam.handler.standard.pdf.PdfHandler.getImage(PdfHandler.java:546)
    at com.day.cq.dam.core.impl.cache.CQBufferedImageCache.getImage(CQBufferedImageCache.java:116)
    at com.day.cq.dam.core.impl.gfx.CommonsGfxRenderer.createImageLayer(CommonsGfxRenderer.java:460)
    at com.day.cq.dam.core.impl.gfx.CommonsGfxRenderer.createLayer(CommonsGfxRenderer.java:138)
    at com.day.cq.dam.core.impl.gfx.CommonsGfxRenderer.render(CommonsGfxRenderer.java:91)
    at com.adobe.cq.gfx.impl.GfxImpl.render(GfxImpl.java:94)
    at com.day.cq.dam.core.impl.RenditionMakerImpl$PlanBasedTemplate.apply(RenditionMakerImpl.java:127)
    at com.day.cq.dam.core.impl.RenditionMakerImpl.generateRenditions(RenditionMakerImpl.java:184)
    at com.day.cq.dam.core.process.CreateThumbnailProcess.createThumbnails(CreateThumbnailProcess.java:119)
    at com.day.cq.dam.core.process.ThumbnailProcess.execute(ThumbnailProcess.java:76)
    at com.day.cq.workflow.compatibility.CQWorkflowProcessRunner.execute(CQWorkflowProcessRunner.java:93)
    at com.adobe.granite.workflow.core.job.HandlerBase.executeProcess(HandlerBase.java:184)
    at com.adobe.granite.workflow.core.job.JobHandler.process(JobHandler.java:234)
    at org.apache.sling.event.impl.jobs.JobConsumerManager$JobConsumerWrapper.process(JobConsumerManager.java:512)
    at org.apache.sling.event.impl.jobs.queues.JobQueueImpl.startJob(JobQueueImpl.java:303)
    - locked <0x00000005cd6dab40> (a org.apache.sling.event.impl.jobs.queues.JobExecutionContextImpl)
    at org.apache.sling.event.impl.jobs.queues.JobQueueImpl.access$100(JobQueueImpl.java:66)
    at org.apache.sling.event.impl.jobs.queues.JobQueueImpl$1.run(JobQueueImpl.java:238)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)

Environment

AEM 6.x using JDK 8

Cause

It is a known issue with PDF processing in AEM (CQ-4202164) caused by an Oracle JVM issue.

Resolution

On Windows and Mac systems using Oracle JVM:

If you are using Oracle JVM on Windows or Mac, configure Java to use the legacy KCMS API instead of LCMS. There is a known issue that makes this fail to work in JDK8 on Linux.

Modify your AEM start script and add this JVM parameter to the CQ_JVM_OPTS variable:

-Dsun.java2d.cmm=sun.java2d.cmm.kcms.KcmsServiceProvider

For example (snippet of start script):

# default JVM options
if [ -z "$CQ_JVM_OPTS" ]; then
        CQ_JVM_OPTS='-server -Xmx1024m -XX:MaxPermSize=256M -Djava.awt.headless=true -Dsun.java2d.cmm=sun.java2d.cmm.kcms.KcmsServiceProvider'
fi

Other Solutions (All Operating Systems)

A. Clear out the update asset workflow sling jobs

Go to http://host:port/crx/de/index.jsp and log in as admin user.
Browse to this path where {slingid} is the sling id of the AEM instance
/var/eventing/jobs/assigned/{slingid}/com.adobe.granite.workflow.job.etc.workflow.models.dam.update_asset.jcr_content.model
Right click and delete the node com.adobe.granite.workflow.job.etc.workflow.models.dam.update_asset.jcr_content.model
Save.
Restart AEM.

This deletes all the active sling jobs for PDFs that are crashing AEM.

B. Install the PDF Rasterizer command-line tool

Download this zip file [1] to your AEM server.
Unzip the zip file to a new folder on the server.
Upload a PDF file to the server in the same folder as where you extracted the zip file.

Run this command in that folder to make sure that the rasterizer works (replace pdffilename.pdf with the filename of the PDF document).

/opt/aem/author62/rasterizer/PDFRasterizer -d -p 1 -s 1280 -t PNG -i pdffilename.pdf

Output would be something like below:

Total time in image conversion (in ms): 163
Total time (in ms): 896. Time to initialize: 12. Time for conversion: 884

Assuming the rasterizer works OK on the command line, follow the steps in this article to configure it to work in AEM.
In addition to the steps on that page, you must also modify the DAM Update Asset workflow step Rasterize PDF/AI Image Preview Rendition. Remove application/pdf from the list of MIME Types.
Click Save to save the workflow model.
Now try uploading PDFs and the renditions are generated fast and without high memory utilization.