Assets Benchmark Guide

Pesquisar
Experience Manager 6.2 Managing Guia do Usuário

Nesta página

Aplica-se a: Experience Manager 6.2 Managing

某些 Creative Cloud 应用程序、服务和功能在中国不可用。

Você está visualizando o conteúdo de ajuda da versão:

This section describes the detailed configuration and test methodology used in the reported benchmarks. Using this detailed information, it should be possible for anyone to re-use or adapt the benchmark procedures in order to compare performance in new and/or different environments.

Detailed Software Configurations

This section describes the detailed configuration of CQ 5.5 DAM and related components that are used in the benchmarks.

CQ Server Configurations

Disable TAR PM Optimization

CRX automatically runs Tar PM Optimization between 2 am and 5 am. If benchmarks are run when TAR PM is running in the background, results might not be as expected. So this scheduled triggering of TAR PM was turned off during the benchmark runs.

Tar PM Optimization runs in two locations:

  • for the workspace
  • for the version store

Observação:

Optimization also runs on the tarJournal directory, but you cannot disable this optimization. You will always see this in the log file.

However, optimization of the tarJournal directory is minimal and will not affect performance.

To disable Tar PM Optimization in both the workspace and the version store:

  1. Edit repository/<your_workspace>/workspace.xml.

    This disables Tar PM Optimization for the workspace.

  2. Make the following changes to the Persistence Manager:

    <PersistenceManager class="com.day.crx.persistence.tar.TarPersistenceManager">
  <param name="autoOptimizeAt" value="-0" />
</PersistenceManager>

  3. Make the same change in  repository/repository.xml

    This disables Tar PM Optimization for the version store.

Change Number of Workflow Threads

  1. Navigate to the URL  http://localhost:4502/system/console/slingevent.

  2. Click Edit for the configuration named Job Queue Configuration: CQ Workflows.

  3. Set Max Parallel Jobs to the required value.

    chlImage_1

  4. Click Save.

FFmpeg Configurations

Modify Codec required for FFmpeg

Modify the audio codec and video codec according to the server platform and FFmpeg build used. For example, for FFmpeg-0.10.2-54.el5.x86_64.rpm to work on RHEL 5.5, audio codec needs to be changed to alac.

Follow the steps below to make the above change:

  1. Navigate to http://localhost:4502/etc/dam/video/hq.html.

  2. Click Edit.

  3. Open the Audio tab.

  4. Set Codec to alac.

    chlImage_1

  5. Click OK.

FFmpeg Process Threads

By default, the maximum number of threads spawned by each FFmpeg process is equal to the number of cpu cores available on the server. To limit the number of threads launched by each FFmpeg process, follow the below steps:

  1. Navigate to http://localhost:4502/etc/dam/video/hq.html.

  2. Click Edit.

  3. Open the Advanced tab under Video Profile Settings.

  4. Add the argument –threads <x> in the Custom FFmpeg arguments text box where <x> is number of threads launched by FFmpeg.

    chlImage_1

  5. Click OK to save. 

Set FFmpeg Encoding to One Pass

By default, Two pass encoding is enabled for FFmpeg. To change it to One pass encoding, follow the steps below:

  1. Navigate to   http://localhost:4502/etc/dam/video/hq.html.

  2. Click Edit.

  3. Open the Advanced tab under Video Profile Settings.

  4. Uncheck the checkbox for Two pass encoding.

    chlImage_1

  5. Click OK.

Test Execution Steps

Observação:

Click the following link to download DAM-capacity-assets.zip.

This archive contains files you will need in order to do the next set of tests.

Download

Obter arquivo
DAM-capacity-assets.zip

Setting up Base Load

  1. Configure your CQ instance with the settings described in Server Configurations and FFmpeg Configuration.

  2. Upload a base load of assets (6K) to the server by running the script UploadLoad.sh:

    UploadLoad.sh <server IP> <server port> <CQ username> <CQ Password>

  3. Upload assets to ViewFolder by running the script UploadViewFolder.sh:

    UploadViewFolder.sh <server IP> <server port> <CQ username> <CQ Password>

     

  4. Run tar optimization and index merging on the base load.

Execute Upload Tests

Test Execution

  1. Install Apache JMeter on your client machine.

  2. Copy DamCapacityUtil.jar to apache-JMeter-2.6\lib\ext.

  3. Create test collateral files using the CreateTestCollateral.sh script. These files will be uploaded into CQ by the JMeter script (UploadAssets.jmx). This determines the upload throughput of the server. Please note that total number of files to be created as part of test collateral depends on the number of transactions to be run. If the number of iterations per thread is 10 and the number of threads is 15, then total transactions would be 10*15=150. Each transaction contains 15 JPG, 4 TIFF and 1 MP4 files. For 150 transactions, the test collateral required would be 2250 JPG, 600 TIFF and 150 MP4 files. 

    Benchmark tests have been run with 150 transactions.

  4. Modify User parameters in the UploadAssets.jmx script as appropriate for your CQ instance.

  5. Execute the JMeter script.

  6. When the test ends, open the resulting CSV file and ensure that all invocations have resulted in success.

Throughput Calculation

Calculation of Scenario Execution Throughput

  1. Find the start time as specified in the first line in the resultant CSV file.

  2. Find the end time from the last invocation in the resulting CSV file.

  3. Let total transactions be T where:

    T = numOfIterations * numOfThreads

  4. Scenario Execution throughput will be:

    T/(End Time - Start Time)/1000 (in transactions per second)

Calculation of Overall Throughput

  1. To keep track of whether any workflows are in progress keep monitoring:

    http://localhost:4502/system/console/slingevent

    Note the time when all workflows finish. Convert this time into epoch (milliseconds) format.

  2. Overall throughput is:

    T/(Workflow End Time - Script Start Time)/1000 (in transactions per second)

Execute Read Only Scenario Tests

The attached sample script (ReadOnlyScenario.jmx) contains recordings of read and search operations done on the created base load and are thus dependent on the created load. To test similar operations on a new load, you need to re-record these operations and replace recordings at appropriate places in the scripts.

Script Recording

Record READ Operation

  1. Clear the browser cache.

  2. Login into CQ.

  3. Open the DAM console:

    http://localost:4502/damadmin

  4. Start recording in JMeter.

  5. In the DAM console, open the folder:

    Digital Assets/capacity

  6. In the DAM console, open the folder:

    Digital Assets/capacity/Folder1

  7. In the DAM console, open the folder:

    Digital Assets/capacity/Folder1/ViewFolder

  8. Stop JMeter recording.

  9. Replace the above recording calls into the JMeter element:

    RunTests-> Reach-ViewFolder - Complete set of requests.

Now start recording read operations for individual files present in the folder:

Digital Assets/capacity/Folder1/ViewFolder

  1. Start recording in JMeter.

  2. In the DAM console, open Test1.jpg (double-click).

  3. Double click on the original icon in the right pane so that the original image is downloaded.

  4. Stop JMeter recording.

  5. Replace the above recordings into the JMeter element:

    RunTests->View Jpg->View File1

  6. Clear the browser cache.

Follow the above steps for all 15 files present in ViewFolder and replace at appropriate places in script.

Record Search Operation

Before recording any search operation, identify 5 searches of each type (TYPE1, TYPE2 and TYPE3) to represent a good mixture of common search use cases.

  • TYPE1: Search with only tags as constraints; for example, search on all assets  tagged with Business keyword (mark checkbox for Business under Tags while recording).
  • TYPE 2 : Search containing ANDing of tags and full text search and file types; for example search on images containing pistol as full text property and are tagged with Business keyword (mark checkbox for Images under File Types, Business under Tags while recording and give pistol in text box for Fulltext).
  • TYPE3: Search containing ANDing of tags and full text search type; for example search on assets containing pistol as full text property and are tagged with Business keyword (mark checkbox for Business under Tags while recording and give pistol in text box for Fulltext).

To record scripts, follow below steps:

  1. Clear the browser cache.

  2. Log into CQ instance.

  3. Open the DAM console:

    http://localhost:4502/damadmin

  4. Start recording in JMeter.

  5. In the DAM console, open the Search tab.

  6. Stop JMeter recording.

  7. Replace the above recording calls in the JMeter element named:

    RunTests-> Search Tags->ReachSearchUI

Now start recording for search operations for the 15 identified searches. The below example covers search recording for TYPE2 search.

  1. Clear the browser cache.

  2. Start recording in JMeter.

  3. In Search of the DAM console, type the search word in Fulltext keyword.

  4. Select the checkbox for File Types to be searched.

  5. Select the checkbox for Tags to be searched.

  6. Click on Search button.

  7. Wait for search to return results.

  8. Stop JMeter recording.

  9. Replace the above recording calls into the JMeter element:

    RunTests-> SearchTagsFTSAndFileType-> Random Controller->Search Query

Follow the above steps for all 15 identified search operations and replace the recordings at appropriate places in script.

Each search query that has been recorded needs to be modified. This is done to ensure that the number of results are not altered by data added by tests. To achieve this an extra AND clause needs to be added to each search query operation.

For example, suppose the recording is done to search all assets tagged with a business keyword. When this query is run on base load of 6K assets, the query returns x results. When the tests for MixedScenario is running when data is added, together with search operations, this same query might return x + y results. So we need to make sure the query always returns the same number of results by modifying the query. We had tagged the complete load of 6K by a unique tag (forum:topic/unanswered), which is not applied to any asset that is uploaded by the test. This unique tag can be utilized to ensure that all search results belong to the 6K load and nothing else. This can be achieved by changing the query to search on assets tagged with Business keyword AND (forum:topic/unanswered) keyword.

So for all the search recordings done, modify all HTTP requests to /bin/querybuilder.json by adding following parameters to the request:

Parameter Value
4_group.property
 jcr:content/metadata/cq:tags
4_group.p.or false
4_group.property.1_value forum:topic/unanswered

Once all these changes have been made to the script, we are ready to execute tests.

Test Execution

  1. Modify user parameters in ReadOnlyScenario.jmx script according to your CQ instance.

  2. Execute JMeter script.

  3. When the test ends, open the resulting CSV file and ensure that all invocations have resulted in success.

Throughput Calculation

Calculation of Overall Throughput

  1. Find the start time as specified in the first line of the resultant CSV file.

  2. Find the end time from the last invocation in the CSV file.

  3. Let total transactions be T where:

    T = numOfIterations * numOfThreads

  4. Overall throughput is:

    T/(End Time - Start Time)/1000 (in transactions per second)

Execute Mixed Scenario Tests

Mixed scenario tests contain upload, read and search operations. Unlike the pure upload tests, where upload and tagging is implemented as simple single POST calls, operations here are recorded using JMeter and then parameterized. Therefore they contain a complete set of GET and POST calls that would be executed if an upload/tag operation is made via the DAM console. Each transaction contains 1 upload and logic is maintained in the script so that the overall upload is done with a ratio of 75% JPG, 20% TIFF and 5% MP4 files.

For READ and SEARCH operations, recordings done for the Read Only Scenario have been utilized so that you will have to replace recordings for Read and Search operations in MixedScenario.jmx too.

Test Execution

  1. Replace recordings for read search operations in the script.

  2. Modify user parameters in MixedScenario.jmx script according to your CQ instance.

  3. Execute the JMeter script.

  4. When the test ends, open the resulting CSV file and ensure that all invocations have resulted in success.

Throughput Calculation

Calculation of Scenario Execution Throughput

  1. Find the start time, as specified in the first line of the resultant CSV file.

  2. Find the end time from the last invocation in the CSV file.

  3. Let total transactions be T where:

    T = numOfIterations * numOfThreads

  4. Scenario Execution throughput is:

    T/(End Time - Start Time)/1000 (in transactions per second)

Calculation of Overall Throughput

  1. To keep track of whether any workflows are in progress keep monitoring:

    http://localhost:4502/system/console/slingevent

    Note the time when all workflows are completed. Convert this time to epoch (milliseconds) format.

  2. Overall throughput is:

    T/(Workflow End Time - Start Time)/1000*60 (transactions per second)