Slow response times and high heap utilization due to full text search

Issue

AEM is running out of memory and crashing due to long running searches. Thread dumps show threads similar to the one below:

"192.168.1.50 [1488891244417] GET /content/geometrixx/en.html HTTP/1.1" #125856 prio=5 os_prio=0 tid=0x00007f32fd3e7800 nid=0x4eb9 runnable [0x00007f3287b64000]
java.lang.Thread.State: RUNNABLE
at java.lang.System.identityHashCode(Native Method)
at org.apache.lucene.index.IndexReader.hashCode(IndexReader.java:290)
at java.util.WeakHashMap.hash(WeakHashMap.java:298)
at java.util.WeakHashMap.put(WeakHashMap.java:449)
at java.util.Collections$SetFromMap.add(Collections.java:5461)
at java.util.Collections$SynchronizedCollection.add(Collections.java:2035)
- locked <0x000000031c1174e8> (a java.util.Collections$SynchronizedSet)
at org.apache.lucene.index.IndexReader.registerParentReader(IndexReader.java:128)
at org.apache.lucene.index.FilterAtomicReader.<init>(FilterAtomicReader.java:334)
at org.apache.lucene.search.highlight.WeightedSpanTermExtractor$DelegatingAtomicReader.<init>(WeightedSpanTermExtractor.java:378)
at org.apache.lucene.search.highlight.WeightedSpanTermExtractor.getLeafContext(WeightedSpanTermExtractor.java:364)
at org.apache.lucene.search.highlight.WeightedSpanTermExtractor.extract(WeightedSpanTermExtractor.java:216)
at org.apache.lucene.search.highlight.WeightedSpanTermExtractor.extract(WeightedSpanTermExtractor.java:99)
at org.apache.lucene.search.highlight.WeightedSpanTermExtractor.getWeightedSpanTerms(WeightedSpanTermExtractor.java:474)
at org.apache.lucene.search.highlight.QueryScorer.initExtractor(QueryScorer.java:217)
at org.apache.lucene.search.highlight.QueryScorer.init(QueryScorer.java:186)
at org.apache.lucene.search.highlight.Highlighter.getBestTextFragments(Highlighter.java:197)
at org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndex.getExcerpt(LuceneIndex.java:496)
at org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndex.access$300(LuceneIndex.java:152)
at org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndex$1.loadDocs(LuceneIndex.java:369)
at org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndex$1.computeNext(LuceneIndex.java:293)
at org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndex$1.computeNext(LuceneIndex.java:283)
at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
at org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndex$LucenePathCursor$1.hasNext(LuceneIndex.java:1116)
at com.google.common.collect.Iterators$7.computeNext(Iterators.java:645)
at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
at org.apache.jackrabbit.oak.spi.query.Cursors$PathCursor.hasNext(Cursors.java:205)
at org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndex$LucenePathCursor.hasNext(LuceneIndex.java:1137)
at org.apache.jackrabbit.oak.plugins.index.aggregate.AggregationCursor.fetchNext(AggregationCursor.java:89)
at org.apache.jackrabbit.oak.plugins.index.aggregate.AggregationCursor.hasNext(AggregationCursor.java:76)
at org.apache.jackrabbit.oak.spi.query.Cursors$IntersectionCursor.fetchNext(Cursors.java:402)
at org.apache.jackrabbit.oak.spi.query.Cursors$IntersectionCursor.hasNext(Cursors.java:394)
at org.apache.jackrabbit.oak.spi.query.Cursors$IntersectionCursor.fetchNext(Cursors.java:402)
at org.apache.jackrabbit.oak.spi.query.Cursors$IntersectionCursor.hasNext(Cursors.java:394)
at org.apache.jackrabbit.oak.query.ast.SelectorImpl.next(SelectorImpl.java:407)
at org.apache.jackrabbit.oak.query.QueryImpl$RowIterator.fetchNext(QueryImpl.java:773)
at org.apache.jackrabbit.oak.query.QueryImpl$RowIterator.hasNext(QueryImpl.java:798)
at com.google.common.collect.Iterators$5.hasNext(Iterators.java:542)
at org.apache.jackrabbit.oak.query.FilterIterators$DistinctIterator.fetchNext(FilterIterators.java:137)
at org.apache.jackrabbit.oak.query.FilterIterators$DistinctIterator.hasNext(FilterIterators.java:151)
at org.apache.jackrabbit.oak.jcr.query.QueryResultImpl$1.fetch(QueryResultImpl.java:108)
at org.apache.jackrabbit.oak.jcr.query.QueryResultImpl$1.<init>(QueryResultImpl.java:104)
at org.apache.jackrabbit.oak.jcr.query.QueryResultImpl.getRows(QueryResultImpl.java:91)
at com.day.cq.search.impl.builder.QueryImpl.execute(QueryImpl.java:475)
at com.day.cq.search.impl.builder.QueryImpl.getResult(QueryImpl.java:106)
at com.day.cq.search.impl.SimpleSearchImpl.getResult(SimpleSearchImpl.java:179)
at com.day.cq.wcm.foundation.Search.getResult(Search.java:219)

Environment

AEM 6.0 and 6.1

Cause

Known limitation with the older style of fulltext lucene index in AEM6.0 and 6.1.

Resolution

Upgrade to AEM6.2 or later version to avoid this issue.

 Adobe

Get help faster and easier

New user?

Adobe MAX 2024

Adobe MAX
The Creativity Conference

Oct 14–16 Miami Beach and online

Adobe MAX

The Creativity Conference

Oct 14–16 Miami Beach and online

Adobe MAX 2024

Adobe MAX
The Creativity Conference

Oct 14–16 Miami Beach and online

Adobe MAX

The Creativity Conference

Oct 14–16 Miami Beach and online