Deleting or moving folders or assets in AEM Assets gets slow or fails

Issue

Deleting or moving certain folders or assets in AEM via the /assets.html or /damadmin UI is either slow or fails.

If the delete or move operation fails, then in the log file you see this [1] WARN log message, "The query read or traversed more than X nodes."

java.lang.UnsupportedOperationException: The query read or traversed more than 100000 nodes. To avoid affecting other tasks, processing was stopped.
at org.apache.jackrabbit.oak.query.FilterIterators.checkReadLimit(FilterIterators.java:66)
at org.apache.jackrabbit.oak.plugins.index.property.strategy.ContentMirrorStoreStrategy$PathIterator.fetchNextPossiblyDuplicate(ContentMirrorStoreStrategy.java:422)
at org.apache.jackrabbit.oak.plugins.index.property.strategy.ContentMirrorStoreStrategy$PathIterator.fetchNext(ContentMirrorStoreStrategy.java:365)
at org.apache.jackrabbit.oak.plugins.index.property.strategy.ContentMirrorStoreStrategy$PathIterator.next(ContentMirrorStoreStrategy.java:449)
at org.apache.jackrabbit.oak.plugins.index.property.strategy.ContentMirrorStoreStrategy$PathIterator.next(ContentMirrorStoreStrategy.java:303)
at com.google.common.collect.Iterators$7.computeNext(Iterators.java:646)
at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
at org.apache.jackrabbit.oak.spi.query.Cursors$PathCursor.hasNext(Cursors.java:205)
at org.apache.jackrabbit.oak.query.ast.SelectorImpl.next(SelectorImpl.java:407)
at org.apache.jackrabbit.oak.query.QueryImpl$RowIterator.fetchNext(QueryImpl.java:773)
at org.apache.jackrabbit.oak.query.QueryImpl$RowIterator.hasNext(QueryImpl.java:798)
at com.google.common.collect.Iterators$5.hasNext(Iterators.java:542)
at org.apache.jackrabbit.oak.query.FilterIterators$DistinctIterator.fetchNext(FilterIterators.java:137)
at org.apache.jackrabbit.oak.query.FilterIterators$DistinctIterator.hasNext(FilterIterators.java:151)
at org.apache.jackrabbit.oak.jcr.query.QueryResultImpl$3.fetch(QueryResultImpl.java:181)
at org.apache.jackrabbit.oak.jcr.query.QueryResultImpl$3.<init>(QueryResultImpl.java:176)
at org.apache.jackrabbit.oak.jcr.query.QueryResultImpl.getNodes(QueryResultImpl.java:170)
at org.apache.sling.jcr.resource.internal.helper.jcr.JcrResourceProvider.findResources(JcrResourceProvider.java:346)
at org.apache.sling.resourceresolver.impl.tree.RootResourceProviderEntry$2.seek(RootResourceProviderEntry.java:148)
at org.apache.sling.resourceresolver.impl.tree.RootResourceProviderEntry$2.<init>(RootResourceProviderEntry.java:134)
at org.apache.sling.resourceresolver.impl.tree.RootResourceProviderEntry.findResources(RootResourceProviderEntry.java:132)
at org.apache.sling.resourceresolver.impl.ResourceResolverImpl.findResources(ResourceResolverImpl.java:694)
at com.day.cq.dam.core.impl.AssetReferenceResolverImpl.getReferences(AssetReferenceResolverImpl.java:524)
at com.day.cq.wcm.core.impl.commands.DeletePageCommand.performCommand(DeletePageCommand.java:142)
at com.day.cq.wcm.core.impl.commands.WCMCommandServlet$CommandHolder.performCommand(WCMCommandServlet.java:161)
at com.day.cq.wcm.core.impl.commands.WCMCommandServlet.performCommand(WCMCommandServlet.java:264)
at com.day.cq.commons.servlets.AbstractCommandServlet.doPost(AbstractCommandServlet.java:49)

Cause

During certain actions in AEM Assets, the system will perform the search below originating from AssetReferenceResolverImpl.getReferences:

//element(*, nt:unstructured)[(jcr:like(@dam:resolvedPath, '/content/dam/geometrixx/services%') or jcr:like(@sling:resource , '/content/dam/geometrixx/services%'))]

When the system has a lot of content then that search doesn't always perform well and could fail with the error above.  

Resolution

To solve this, we can add some properties to the out-of-the-box /oak:index/ntBaseLucene index.

  1. Go to http://aem-host:port/crx/de/index.jsp and log in as admin.

  2. Browse to /oak:index/ntBaseLucene/indexRules/nt:base/properties

  3. Add two nt:unstructured nodes slingResource and damResolvedPath under /oak:index/ntBaseLucene/indexRules/nt:base/properties.

  4. Set the properties below on the nodes (where ordered and propertyIndex properties are of Boolean type):

    slingResource
      name="sling:resource"
      ordered=false
      propertyIndex= true
      type="String"
    damResolvedPath
      name="dam:resolvedPath"
      ordered=false
      propertyIndex=true
      type="String"

  5. On the /oak:index/ntBaseLucene node, set the property reindex=true.

  6. Click Save All.

  7. Monitor the error.log to see when indexing is completed:

    Reindexing completed for indexes: [/oak:index/ntBaseLucene]

    You can also see that indexing is completed by refreshing the /oak:index/ntBaseLucene node in CRXDe as the reindex property would go back to false.

  8. Once indexing is completed then go back to CRXDe and set the "type" property value to "disabled" on these two indexes:

    • /oak:index/slingResource
    • /oak:index/damResolvedPath
  9. Click "Save All"

    Now folder and asset delete and move operations should run faster and be more stable.