Troubleshooting Oak Startup Issues

Search

Issue

When starting AEM, the error below is observed and AEM fails to start up: 

*ERROR* [FelixStartLevel] org.apache.jackrabbit.oak-core [org.apache.jackrabbit.oak.plugins.segment.SegmentNodeStoreService(87)] The activate method has thrown an exception (java.lang.IllegalStateException: RefId '7' doesn't exist in data segment 754dbe96-ad7f-4e58-a504-df1103bae4df. Creation date delta is 21 ms.)
java.lang.IllegalStateException: RefId '7' doesn't exist in data segment 754dbe96-ad7f-4e58-a504-df1103bae4df. Creation date delta is 21 ms.
    at org.apache.jackrabbit.oak.plugins.segment.Segment.getRefId(Segment.java:230)
    at org.apache.jackrabbit.oak.plugins.segment.Segment.internalReadRecordId(Segment.java:342)
    at org.apache.jackrabbit.oak.plugins.segment.Segment.readRecordId(Segment.java:338)
    at org.apache.jackrabbit.oak.plugins.segment.SegmentNodeState.getTemplateId(SegmentNodeState.java:70)
    at org.apache.jackrabbit.oak.plugins.segment.SegmentNodeState.getTemplate(SegmentNodeState.java:79)
    at org.apache.jackrabbit.oak.plugins.segment.SegmentNodeState.getChildNode(SegmentNodeState.java:381)
    at org.apache.jackrabbit.oak.plugins.segment.SegmentNodeStore.getRoot(SegmentNodeStore.java:146)
    at org.apache.jackrabbit.oak.plugins.segment.SegmentNodeStore.<init>(SegmentNodeStore.java:98)
    at org.apache.jackrabbit.oak.plugins.segment.SegmentNodeStoreService.registerSegmentStore(SegmentNodeStoreService.java:333)
    at org.apache.jackrabbit.oak.plugins.segment.SegmentNodeStoreService.registerNodeStore(SegmentNodeStoreService.java:236)
    at org.apache.jackrabbit.oak.plugins.segment.SegmentNodeStoreService.activate(SegmentNodeStoreService.java:231)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.felix.scr.impl.helper.BaseMethod.invokeMethod(BaseMethod.java:222)
    at org.apache.felix.scr.impl.helper.BaseMethod.access$500(BaseMethod.java:37)
    at org.apache.felix.scr.impl.helper.BaseMethod$Resolved.invoke(BaseMethod.java:615)
    at org.apache.felix.scr.impl.helper.BaseMethod.invoke(BaseMethod.java:499)
    at org.apache.felix.scr.impl.helper.ActivateMethod.invoke(ActivateMethod.java:295)
    at org.apache.felix.scr.impl.manager.SingleComponentManager.createImplementationObject(SingleComponentManager.java:302)
    at org.apache.felix.scr.impl.manager.SingleComponentManager.createComponent(SingleComponentManager.java:113)
    at org.apache.felix.scr.impl.manager.SingleComponentManager.getService(SingleComponentManager.java:832)
    at org.apache.felix.scr.impl.manager.SingleComponentManager.getServiceInternal(SingleComponentManager.java:799)
    at org.apache.felix.scr.impl.manager.AbstractComponentManager.activateInternal(AbstractComponentManager.java:724)
    at org.apache.felix.scr.impl.manager.AbstractComponentManager.enable(AbstractComponentManager.java:399)
    at org.apache.felix.scr.impl.config.ConfigurableComponentHolder.enableComponents(ConfigurableComponentHolder.java:669)
    at org.apache.felix.scr.impl.BundleComponentActivator.initialEnable(BundleComponentActivator.java:184)
    at org.apache.felix.scr.impl.Activator.loadComponents(Activator.java:332)
    at org.apache.felix.scr.impl.Activator.access$000(Activator.java:49)
    at org.apache.felix.scr.impl.Activator$ScrExtension.start(Activator.java:257)
    at org.apache.felix.utils.extender.AbstractExtender.createExtension(AbstractExtender.java:259)
    at org.apache.felix.utils.extender.AbstractExtender.modifiedBundle(AbstractExtender.java:232)
    at org.osgi.util.tracker.BundleTracker$Tracked.customizerModified(BundleTracker.java:482)
    at org.osgi.util.tracker.BundleTracker$Tracked.customizerModified(BundleTracker.java:415)
    at org.osgi.util.tracker.AbstractTracked.track(AbstractTracked.java:232)
    at org.osgi.util.tracker.BundleTracker$Tracked.bundleChanged(BundleTracker.java:444)
    at org.apache.felix.framework.util.EventDispatcher.invokeBundleListenerCallback(EventDispatcher.java:913)
    at org.apache.felix.framework.util.EventDispatcher.fireEventImmediately(EventDispatcher.java:834)
    at org.apache.felix.framework.util.EventDispatcher.fireBundleEvent(EventDispatcher.java:516)
    at org.apache.felix.framework.Felix.fireBundleEvent(Felix.java:4531)
    at org.apache.felix.framework.Felix.startBundle(Felix.java:2169)
    at org.apache.felix.framework.Felix.setActiveStartLevel(Felix.java:1368)
    at org.apache.felix.framework.FrameworkStartLevelImpl.run(FrameworkStartLevelImpl.java:308)
    at java.lang.Thread.run(Thread.java:745)

Environment

AEM 6.1

Resolution

  1. Browse to http://hostname:port/system/console/bundles and log in as admin (note that you may need to log in using admin / admin instead of your usual password as Oak repository is unavailable).

  2. Search for bundle "oak-core" and take note of the version of the bundle.

  3. Download the version of oak-run jar file that matches your version of Oak [1] (e.g: if your oak-core bundle is 1.2.18, you need oak-run-1.2.18.jar).

  4. Place the oak-run jar file on the AEM server in the same directory as the AEM jar file and crx-quickstart folder.

  5. Stop AEM.

  6. Back up all files in /crx-quickstart/repository/segmentstore/.

Execute the repair process:

  1. Start by running consistency check.  It will search backwards through the repository revisions until it finds a consistent one: 

    $> java -jar oak-run-*.jar check -d1 --bin=-1 -p crx-quickstart/repository/segmentstore/
    14:00:30.783 [main] INFO o.a.j.o.p.s.f.t.ConsistencyChecker - Found latest good revision afdb922d-ba53-4a1b-aa1b-1cb044b535cf:234880

  2. Revert the repository to the good revision by editing ./crx-quickstart/repository/segmentstore/journal.log and deleting all lines after the line containing the latest good revision.

    WARNING: If the last good revision happens to be an old one then this process may roll back your data to a much older date and time.

  3. Remove all ./crx-quickstart/repository/segmentstore/*.bak files.

  4. Run checkpoint clean-up to remove orphaned checkpoints: 

    java -jar oak-run-*.jar checkpoints ./crx-quickstart/repository/segmentstore rm-unreferenced

  5. Run offline compaction on the repository.

    java -jar oak-run-*.jar compact ./crx-quickstart/repository/segmentstore/

 Adobe

Get help faster and easier

New user?

Quick links

View all your plans Manage your plans
View quick links
Hide quick links
Adobe MAX 2024

Adobe MAX
The Creativity Conference

Oct 14–16 Miami Beach and online

Register Now

Adobe MAX

The Creativity Conference

Oct 14–16 Miami Beach and online

Register Now
Adobe MAX 2024

Adobe MAX
The Creativity Conference

Oct 14–16 Miami Beach and online

Register Now

Adobe MAX

The Creativity Conference

Oct 14–16 Miami Beach and online

Register Now

Legal Notices    |    Online Privacy Policy