This section describes the detailed configuration and test methodology used in the reported benchmarks. Using this detailed information, it should be possible for anyone to re-use or adapt the benchmark procedures in order to compare performance in new and/or different environments.
This section describes the detailed configuration of CQ 5.5 DAM and related components that are used in the benchmarks.
CRX automatically runs Tar PM Optimization between 2 am and 5 am. If benchmarks are run when TAR PM is running in the background, results might not be as expected. So this scheduled triggering of TAR PM was turned off during the benchmark runs.
Tar PM Optimization runs in two locations:
- for the workspace
- for the version store
Optimization also runs on the tarJournal directory, but you cannot disable this optimization. You will always see this in the log file.
However, optimization of the tarJournal directory is minimal and will not affect performance.
Modify the audio codec and video codec according to the server platform and FFmpeg build used. For example, for FFmpeg-0.10.2-54.el5.x86_64.rpm to work on RHEL 5.5, audio codec needs to be changed to alac.
Follow the steps below to make the above change:
By default, the maximum number of threads spawned by each FFmpeg process is equal to the number of cpu cores available on the server. To limit the number of threads launched by each FFmpeg process, follow the below steps:
By default, Two pass encoding is enabled for FFmpeg. To change it to One pass encoding, follow the steps below:
Click the following link to download DAM-capacity-assets.zip.
This archive contains files you will need in order to do the next set of tests.
Create test collateral files using the CreateTestCollateral.sh script. These files will be uploaded into CQ by the JMeter script (UploadAssets.jmx). This determines the upload throughput of the server. Please note that total number of files to be created as part of test collateral depends on the number of transactions to be run. If the number of iterations per thread is 10 and the number of threads is 15, then total transactions would be 10*15=150. Each transaction contains 15 JPG, 4 TIFF and 1 MP4 files. For 150 transactions, the test collateral required would be 2250 JPG, 600 TIFF and 150 MP4 files.
Benchmark tests have been run with 150 transactions.
The attached sample script (ReadOnlyScenario.jmx) contains recordings of read and search operations done on the created base load and are thus dependent on the created load. To test similar operations on a new load, you need to re-record these operations and replace recordings at appropriate places in the scripts.
Now start recording read operations for individual files present in the folder:
Follow the above steps for all 15 files present in ViewFolder and replace at appropriate places in script.
Before recording any search operation, identify 5 searches of each type (TYPE1, TYPE2 and TYPE3) to represent a good mixture of common search use cases.
- TYPE1: Search with only tags as constraints; for example, search on all assets tagged with Business keyword (mark checkbox for Business under Tags while recording).
- TYPE 2 : Search containing ANDing of tags and full text search and file types; for example search on images containing pistol as full text property and are tagged with Business keyword (mark checkbox for Images under File Types, Business under Tags while recording and give pistol in text box for Fulltext).
- TYPE3: Search containing ANDing of tags and full text search type; for example search on assets containing pistol as full text property and are tagged with Business keyword (mark checkbox for Business under Tags while recording and give pistol in text box for Fulltext).
Now start recording for search operations for the 15 identified searches. The below example covers search recording for TYPE2 search.
Follow the above steps for all 15 identified search operations and replace the recordings at appropriate places in script.
Each search query that has been recorded needs to be modified. This is done to ensure that the number of results are not altered by data added by tests. To achieve this an extra AND clause needs to be added to each search query operation.
For example, suppose the recording is done to search all assets tagged with a business keyword. When this query is run on base load of 6K assets, the query returns x results. When the tests for MixedScenario is running when data is added, together with search operations, this same query might return x + y results. So we need to make sure the query always returns the same number of results by modifying the query. We had tagged the complete load of 6K by a unique tag (forum:topic/unanswered), which is not applied to any asset that is uploaded by the test. This unique tag can be utilized to ensure that all search results belong to the 6K load and nothing else. This can be achieved by changing the query to search on assets tagged with Business keyword AND (forum:topic/unanswered) keyword.
So for all the search recordings done, modify all HTTP requests to /bin/querybuilder.json by adding following parameters to the request:
Mixed scenario tests contain upload, read and search operations. Unlike the pure upload tests, where upload and tagging is implemented as simple single POST calls, operations here are recorded using JMeter and then parameterized. Therefore they contain a complete set of GET and POST calls that would be executed if an upload/tag operation is made via the DAM console. Each transaction contains 1 upload and logic is maintained in the script so that the overall upload is done with a ratio of 75% JPG, 20% TIFF and 5% MP4 files.
For READ and SEARCH operations, recordings done for the Read Only Scenario have been utilized so that you will have to replace recordings for Read and Search operations in MixedScenario.jmx too.