This article discusses how to restore missing S3 datastore blobs.

Issue

An error similar to the following is observed in the AEM logs:

27.03.2017 13:43:39.698 *WARN* [qtp120214359-3936] org.apache.jackrabbit.oak.plugins.blob.datastore.DataStoreBlobStore Error occurred while loading bytes from steam while fetching for id 857ee6b8664cea7e092775a2ec22dd6138342800#23435
java.util.concurrent.ExecutionException: java.io.IOException: org.apache.jackrabbit.core.data.DataStoreException: Could not length of dataIdentifier 857ee6b8664cea7e092775a2ec22dd6138342800
at org.apache.jackrabbit.oak.cache.CacheLIRS$Segment.load(CacheLIRS.java:1010)
at org.apache.jackrabbit.oak.cache.CacheLIRS$Segment.get(CacheLIRS.java:971)
at org.apache.jackrabbit.oak.cache.CacheLIRS.get(CacheLIRS.java:282)
at org.apache.jackrabbit.oak.plugins.blob.datastore.DataStoreBlobStore.getInputStream(DataStoreBlobStore.java:298)
at org.apache.jackrabbit.oak.plugins.blob.BlobStoreBlob.getNewStream(BlobStoreBlob.java:47)
at org.apache.jackrabbit.oak.plugins.segment.SegmentBlob.getNewStream(SegmentBlob.java:265)
at org.apache.jackrabbit.oak.plugins.segment.SegmentBlob.getNewStream(SegmentBlob.java:83)
at org.apache.jackrabbit.oak.plugins.value.ValueImpl.getStream(ValueImpl.java:297)
at org.apache.jackrabbit.vault.fs.PropertyValueArtifact$PVAInputStream.assertOpen(PropertyValueArtifact.java:300)
at org.apache.jackrabbit.vault.fs.PropertyValueArtifact$PVAInputStream.read(PropertyValueArtifact.java:312)
at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1792)
at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1769)
at org.apache.commons.io.IOUtils.copy(IOUtils.java:1744)
at org.apache.jackrabbit.vault.fs.io.JarExporter.writeFile(JarExporter.java:128)
at org.apache.jackrabbit.vault.fs.io.AbstractExporter.export(AbstractExporter.java:216)
at org.apache.jackrabbit.vault.fs.io.AbstractExporter.export(AbstractExporter.java:214)
at org.apache.jackrabbit.vault.fs.io.AbstractExporter.export(AbstractExporter.java:214)
at org.apache.jackrabbit.vault.fs.io.AbstractExporter.export(AbstractExporter.java:214)
at org.apache.jackrabbit.vault.fs.io.AbstractExporter.export(AbstractExporter.java:214)
at org.apache.jackrabbit.vault.fs.io.AbstractExporter.export(AbstractExporter.java:214)
at org.apache.jackrabbit.vault.fs.io.AbstractExporter.export(AbstractExporter.java:184)
at org.apache.jackrabbit.vault.packaging.impl.PackageManagerImpl.assemble(PackageManagerImpl.java:131)
at org.apache.jackrabbit.vault.packaging.impl.PackageManagerImpl.assemble(PackageManagerImpl.java:85)
at org.apache.jackrabbit.vault.packaging.impl.JcrPackageManagerImpl.assemble(JcrPackageManagerImpl.java:471)
at org.apache.jackrabbit.vault.packaging.impl.JcrPackageImpl.snapshot(JcrPackageImpl.java:549)
at org.apache.jackrabbit.vault.packaging.impl.JcrPackageImpl.extract(JcrPackageImpl.java:420)
at org.apache.jackrabbit.vault.packaging.impl.JcrPackageImpl.extract(JcrPackageImpl.java:483)
at org.apache.jackrabbit.vault.packaging.impl.JcrPackageImpl.install(JcrPackageImpl.java:392)
at com.day.jcr.vault.packaging.impl.JrVltJcrPackageAdapter.install(JrVltJcrPackageAdapter.java:107)
at com.day.crx.packaging.impl.J2EEPackageManager.consoleInstall(J2EEPackageManager.java:336)
at com.day.crx.packaging.impl.J2EEPackageManager.doPost(J2EEPackageManager.java:175)
at com.day.crx.packaging.impl.PackageManagerServlet.doPost(PackageManagerServlet.java:152)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:644)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:725)
at org.apache.felix.http.base.internal.handler.ServletHandler.handle(ServletHandler.java:85)
at org.apache.felix.http.base.internal.dispatch.InvocationChain.doFilter(InvocationChain.java:79)
at com.adobe.granite.license.impl.LicenseCheckFilter.doFilter(LicenseCheckFilter.java:308)
at org.apache.felix.http.base.internal.handler.FilterHandler.handle(FilterHandler.java:135)
at org.apache.felix.http.base.internal.dispatch.InvocationChain.doFilter(InvocationChain.java:74)
at org.apache.felix.http.sslfilter.internal.SslFilter.doFilter(SslFilter.java:89)
at org.apache.felix.http.base.internal.handler.FilterHandler.handle(FilterHandler.java:135)
at org.apache.felix.http.base.internal.dispatch.InvocationChain.doFilter(InvocationChain.java:74)
at org.apache.sling.security.impl.ReferrerFilter.doFilter(ReferrerFilter.java:290)
at org.apache.felix.http.base.internal.handler.FilterHandler.handle(FilterHandler.java:135)
at org.apache.felix.http.base.internal.dispatch.InvocationChain.doFilter(InvocationChain.java:74)
at org.apache.sling.i18n.impl.I18NFilter.doFilter(I18NFilter.java:129)
at org.apache.felix.http.base.internal.handler.FilterHandler.handle(FilterHandler.java:135)
at org.apache.felix.http.base.internal.dispatch.InvocationChain.doFilter(InvocationChain.java:74)
at org.apache.sling.featureflags.impl.FeatureManager.doFilter(FeatureManager.java:116)
at org.apache.felix.http.base.internal.handler.FilterHandler.handle(FilterHandler.java:135)
at org.apache.felix.http.base.internal.dispatch.InvocationChain.doFilter(InvocationChain.java:74)
at org.apache.sling.engine.impl.log.RequestLoggerFilter.doFilter(RequestLoggerFilter.java:75)
at org.apache.felix.http.base.internal.handler.FilterHandler.handle(FilterHandler.java:135)
at org.apache.felix.http.base.internal.dispatch.InvocationChain.doFilter(InvocationChain.java:74)
at org.apache.felix.http.base.internal.dispatch.Dispatcher.dispatch(Dispatcher.java:124)
at org.apache.felix.http.base.internal.DispatcherServlet.service(DispatcherServlet.java:61)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:725)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:812)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:587)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
at org.eclipse.jetty.server.Server.handle(Server.java:499)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: org.apache.jackrabbit.core.data.DataStoreException: Could not length of dataIdentifier 857ee6b8664cea7e092775a2ec22dd6138342800
at org.apache.jackrabbit.oak.plugins.blob.datastore.DataStoreBlobStore.getStream(DataStoreBlobStore.java:534)
at org.apache.jackrabbit.oak.plugins.blob.datastore.DataStoreBlobStore$2.call(DataStoreBlobStore.java:302)
at org.apache.jackrabbit.oak.plugins.blob.datastore.DataStoreBlobStore$2.call(DataStoreBlobStore.java:298)
at org.apache.jackrabbit.oak.cache.CacheLIRS$Segment.load(CacheLIRS.java:1006)
... 73 common frames omitted
Caused by: org.apache.jackrabbit.core.data.DataStoreException: Could not length of dataIdentifier 857ee6b8664cea7e092775a2ec22dd6138342800
at org.apache.jackrabbit.oak.blob.cloud.aws.s3.S3Backend.getLength(S3Backend.java:485)
at org.apache.jackrabbit.core.data.CachingDataStore.getLength(CachingDataStore.java:669)
at org.apache.jackrabbit.core.data.CachingDataStore.getRecord(CachingDataStore.java:467)
at org.apache.jackrabbit.oak.plugins.blob.datastore.DataStoreBlobStore.getDataRecord(DataStoreBlobStore.java:543)
at org.apache.jackrabbit.oak.plugins.blob.datastore.DataStoreBlobStore.getStream(DataStoreBlobStore.java:528)
... 76 common frames omitted
Caused by: com.amazonaws.services.s3.model.AmazonS3Exception: Not Found (Service: Amazon S3; Status Code: 404; Error Code: 404 Not Found; Request ID: 84B4B960EBF4F10B)
at com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:1389)
at com.amazonaws.http.AmazonHttpClient.executeOneRequest(AmazonHttpClient.java:902)
at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:607)
at com.amazonaws.http.AmazonHttpClient.doExecute(AmazonHttpClient.java:376)
at com.amazonaws.http.AmazonHttpClient.executeWithTimer(AmazonHttpClient.java:338)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:287)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3826)
at com.amazonaws.services.s3.AmazonS3Client.getObjectMetadata(AmazonS3Client.java:1015)
at com.amazonaws.services.s3.AmazonS3Client.getObjectMetadata(AmazonS3Client.java:991)
at org.apache.jackrabbit.oak.blob.cloud.aws.s3.S3Backend.getLength(S3Backend.java:478)
... 80 common frames omitted

Environment

AEM 6.2 / Oak 1.4.0-1.4.9

Cause

A known bug OAK-5005 is responsible for this issue.

Resolution

  1. To restore the missing blobs, follow the steps to recover missing datastore files.

  2. Upload the same missing blobs to S3 using the "aws s3 cp" command line tool.

  3. Install the latest AEM 6.2 Oak hot fix.

이 작업에는 Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License의 라이센스가 부여되었습니다.  Twitter™ 및 Facebook 게시물은 Creative Commons 약관을 적용받지 않습니다.

법적 고지 사항   |   온라인 개인 정보 보호 정책