Troubleshooting Oak Startup Issues

Otsi

Viimati uuendatud 20. mai 2021

Issue

When starting AEM, the error below is observed and AEM fails to start up:

*ERROR* [FelixStartLevel] org.apache.jackrabbit.oak-core [org.apache.jackrabbit.oak.plugins.segment.SegmentNodeStoreService(87)] The activate method has thrown an exception (java.lang.IllegalStateException: RefId '7' doesn't exist in data segment 754dbe96-ad7f-4e58-a504-df1103bae4df. Creation date delta is 21 ms.)
java.lang.IllegalStateException: RefId '7' doesn't exist in data segment 754dbe96-ad7f-4e58-a504-df1103bae4df. Creation date delta is 21 ms.
    at org.apache.jackrabbit.oak.plugins.segment.Segment.getRefId(Segment.java:230)
    at org.apache.jackrabbit.oak.plugins.segment.Segment.internalReadRecordId(Segment.java:342)
    at org.apache.jackrabbit.oak.plugins.segment.Segment.readRecordId(Segment.java:338)
    at org.apache.jackrabbit.oak.plugins.segment.SegmentNodeState.getTemplateId(SegmentNodeState.java:70)
    at org.apache.jackrabbit.oak.plugins.segment.SegmentNodeState.getTemplate(SegmentNodeState.java:79)
    at org.apache.jackrabbit.oak.plugins.segment.SegmentNodeState.getChildNode(SegmentNodeState.java:381)
    at org.apache.jackrabbit.oak.plugins.segment.SegmentNodeStore.getRoot(SegmentNodeStore.java:146)
    at org.apache.jackrabbit.oak.plugins.segment.SegmentNodeStore.<init>(SegmentNodeStore.java:98)
    at org.apache.jackrabbit.oak.plugins.segment.SegmentNodeStoreService.registerSegmentStore(SegmentNodeStoreService.java:333)
    at org.apache.jackrabbit.oak.plugins.segment.SegmentNodeStoreService.registerNodeStore(SegmentNodeStoreService.java:236)
    at org.apache.jackrabbit.oak.plugins.segment.SegmentNodeStoreService.activate(SegmentNodeStoreService.java:231)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.felix.scr.impl.helper.BaseMethod.invokeMethod(BaseMethod.java:222)
    at org.apache.felix.scr.impl.helper.BaseMethod.access$500(BaseMethod.java:37)
    at org.apache.felix.scr.impl.helper.BaseMethod$Resolved.invoke(BaseMethod.java:615)
    at org.apache.felix.scr.impl.helper.BaseMethod.invoke(BaseMethod.java:499)
    at org.apache.felix.scr.impl.helper.ActivateMethod.invoke(ActivateMethod.java:295)
    at org.apache.felix.scr.impl.manager.SingleComponentManager.createImplementationObject(SingleComponentManager.java:302)
    at org.apache.felix.scr.impl.manager.SingleComponentManager.createComponent(SingleComponentManager.java:113)
    at org.apache.felix.scr.impl.manager.SingleComponentManager.getService(SingleComponentManager.java:832)
    at org.apache.felix.scr.impl.manager.SingleComponentManager.getServiceInternal(SingleComponentManager.java:799)
    at org.apache.felix.scr.impl.manager.AbstractComponentManager.activateInternal(AbstractComponentManager.java:724)
    at org.apache.felix.scr.impl.manager.AbstractComponentManager.enable(AbstractComponentManager.java:399)
    at org.apache.felix.scr.impl.config.ConfigurableComponentHolder.enableComponents(ConfigurableComponentHolder.java:669)
    at org.apache.felix.scr.impl.BundleComponentActivator.initialEnable(BundleComponentActivator.java:184)
    at org.apache.felix.scr.impl.Activator.loadComponents(Activator.java:332)
    at org.apache.felix.scr.impl.Activator.access$000(Activator.java:49)
    at org.apache.felix.scr.impl.Activator$ScrExtension.start(Activator.java:257)
    at org.apache.felix.utils.extender.AbstractExtender.createExtension(AbstractExtender.java:259)
    at org.apache.felix.utils.extender.AbstractExtender.modifiedBundle(AbstractExtender.java:232)
    at org.osgi.util.tracker.BundleTracker$Tracked.customizerModified(BundleTracker.java:482)
    at org.osgi.util.tracker.BundleTracker$Tracked.customizerModified(BundleTracker.java:415)
    at org.osgi.util.tracker.AbstractTracked.track(AbstractTracked.java:232)
    at org.osgi.util.tracker.BundleTracker$Tracked.bundleChanged(BundleTracker.java:444)
    at org.apache.felix.framework.util.EventDispatcher.invokeBundleListenerCallback(EventDispatcher.java:913)
    at org.apache.felix.framework.util.EventDispatcher.fireEventImmediately(EventDispatcher.java:834)
    at org.apache.felix.framework.util.EventDispatcher.fireBundleEvent(EventDispatcher.java:516)
    at org.apache.felix.framework.Felix.fireBundleEvent(Felix.java:4531)
    at org.apache.felix.framework.Felix.startBundle(Felix.java:2169)
    at org.apache.felix.framework.Felix.setActiveStartLevel(Felix.java:1368)
    at org.apache.felix.framework.FrameworkStartLevelImpl.run(FrameworkStartLevelImpl.java:308)
    at java.lang.Thread.run(Thread.java:745)

Environment

AEM 6.1

Resolution

Browse to http://hostname:port/system/console/bundles and log in as admin (note that you may need to log in using admin / admin instead of your usual password as Oak repository is unavailable).
Search for bundle "oak-core" and take note of the version of the bundle.
Download the version of oak-run jar file that matches your version of Oak [1] (e.g: if your oak-core bundle is 1.2.18, you need oak-run-1.2.18.jar).
Place the oak-run jar file on the AEM server in the same directory as the AEM jar file and crx-quickstart folder.
Stop AEM.
Back up all files in /crx-quickstart/repository/segmentstore/.

Execute the repair process:

Start by running consistency check. It will search backwards through the repository revisions until it finds a consistent one:

$> java -jar oak-run-*.jar check -d1 --bin=-1 -p crx-quickstart/repository/segmentstore/
14:00:30.783 [main] INFO o.a.j.o.p.s.f.t.ConsistencyChecker - Found latest good revision afdb922d-ba53-4a1b-aa1b-1cb044b535cf:234880
Revert the repository to the good revision by editing ./crx-quickstart/repository/segmentstore/journal.log and deleting all lines after the line containing the latest good revision.

WARNING: If the last good revision happens to be an old one then this process may roll back your data to a much older date and time.
Remove all ./crx-quickstart/repository/segmentstore/*.bak files.
Run checkpoint clean-up to remove orphaned checkpoints:

java -jar oak-run-*.jar checkpoints ./crx-quickstart/repository/segmentstore rm-unreferenced
Run offline compaction on the repository.

java -jar oak-run-*.jar compact ./crx-quickstart/repository/segmentstore/