Vous consultez actuellement l'aide de la version:

chlimage_1

What is the “Tough Day” test?

The default “Tough Day” test simulates the daily load of around 1000 authors in a worst-case scenario with all the operations going on at the same time. To make things worse the test deviates from best practices using very unfortunate, not recommended content structures.

Remarque :

For information about maximizing response times, see Performance Optimization.

Use cases

Tough Day has mainly two usecases:

  • Stress testing the system with a much larger number of users in parallel than can normally occur
  • Simulate a certain number of users in parallel that is close to the number of peak users an author installation can have. This is the more used test.

Attention :

Please note that for the second use-case, the createPages.thinkt parameter needs to be specified. Example:

java -Xmx1024m -Dhostname=localhost -Dport=4502 -DcreatePages.toplevels=2 -DcreatePages.sublevels=2 -DcreatePages.subsublevels=2 -DcreatePages.concurrentUsers=3 -DcreatePages.rampup=1 -DcreatePages.thinkt=5 -DcreatePages.activate=true -DcreatePages.delete=true -jar toughday.jar createPages 

Remarque :

For estimating the number of users on an author instance, you can use the Request Analysis Script

 

Outline

A cron job triggers a bash script which downloads the current nightly build. The nightly build is uploaded with the S3CMD tools into an Amazon S3 bucket inside the cloud. Multiple EC2 machines with different configurations and hardware specifications are instantiated from existing AMIs. Each machine starts an ANT script which installs and configures the nightly build of CQ and starts all configured tests parallel. After the tests have passed or failed the results are gathered and send back to the Day QaBase. The virtual machines are terminated 48 hours later.

Sub tests

Create pages

Creates content pages inside CQ, adds different components to the page, activates the page and deletes the page. This test is executed with 30 concurrent users and zero think time. The test creates a three level site hierarchy.

Upload image

Imports images into CQ DAM. Uses the ExifTool to modify and multiply one source image.

MSM

Performs many MSM page roll outs, by creating live copies of the Geometrixx Demo Site.

Import User

Imports user into CQ with random IDs and Names.

Tags

Creates a large taxonomy tree inside CQ with 21050 tags.

Assign Tags

Assigns 1000 tags to each Geometrixx Demo page.

Tough Day advantages

  • Hardware independent
  • Persistence Manager independent
  • Cluster independent
  • allows to create height load
  • allows to create large amount of content
  • allows to test with many concurrent users
  • allows to test different kind of content structures
  • allows to test different kind of CQ operations at the same time
  • allows to measure infrastructure performance

Cloud ready

The test was designed to use the cloud for the test execution. The cloud offers the possibility to run the same test on different hardware, OS, java versions and on 32 or 64 bit architecture.

Cluster ready

Can also be used to test the cluster. Configure the hostname of the master or a slave node to run the test against this node. The master and one or more slave nodes can also be tested simultaneously.

Cluster & Cloud

Testing the cluster functionality inside the Cloud allows you to switch between one machine, multiple machines in the same zone and same region, different zones but same region and different regions.

chlimage_1

What does the “Tough Day” test do

The number of pages, users, images etc. that will be created can be configured. Following is the default for our nightly build tests:

chlimage_1

Remarque :

"These are all “write tests," simulating Individual CQ5 end-user browser Operations and cannot be compared to bare-bone repository operations. Tests performed: running Windows 2008 Server”.

10x Tough Day

Once every load, we execute a test that runs ten times longer than our default test.

chlimage_1

Benchmarking

The following results are from a test on a medium amazon machine.

  • Create 1000 pages with 5000 page updates:  225 seconds
  • Import 50 Images: 79 seconds
  • Create 2000 users: 101 seconds
chlimage_1

Hardware

Amazon EC2 m1.large

7.5 GB memory
4 EC2 Compute Units (2 virtual cores with 2 EC2 Compute Units each)
850 GB instance storage
64-bit platform
I/O Performance: High
API name: m1.large

OS

Windows Server 2008 DataCenter 64 Bit

Scalability

chlimage_1

Increasing the CPU power by 26 times increases the throughput by 11 times and with very fast I/0 even by 24 times.

Tools Used

Jmeter

Used for the create page and MSM test.

ExifTool

Used for the upload image test.

Ant

Used as test runner for all tests.

EC2 Tools

Used for starting and stopping virtual machines in the cloud.

S3CMD

Used for uploading and downloading binaries into the cloud.

CURL

Used for the upload image, tags and create user test.

Bash

Used for the upload image, tags and create user test.

Customer Tough Day Tool

Toughday-5.6.1.jar is a command line tool to quickly stress test your system. It is a derivative of the Tough Day test. It allows you to create pages, users, tags and images on a CQ instance. You can measure your server performance with a benchmark.

Example Usage

java -Xmx1024m -Dhostname=localhost -Dport=4502 -DuploadImage.count=10 -jar toughday-5.6.1.jar uploadImage

This would unpack the jar into a toughday folder in the current directory. After that it starts an uploadImage test with 10 images to a CQ instance which is running on localhost:4502.

The tool supports CQ 5.6.1

Important folder and files:

  • toughday.properties
    Contains the default values for the tests
  • tasks
    Contains the tests, each test contains his own folder
  • task/build.fragment
    Which is an ant fragment file which contains at least one ant target with the name of the test e.g. "createPages" and must be depending on the "test" target.

Known Issues

java -Xmx1024m -Dhostname=localhost -Dport=4502 -jar toughday-5.6.1.jar all

This runs all tests in parallel. The addTagsToGeometrixxPages test depends on the tags test so it might fail because of the timing.

Download

The current version of the jar file can be downloaded here:

Telechargement

Remarque :

This JAR file was updated for AEM 6.1 due to the CRSF protection feature. Using the earlier version of this JAR file will result in the following errors:

"com.adobe.granite.csrf.impl.CSRFFilter isValidRequest: empty CSRF token - rejecting"

"com.adobe.granite.csrf.impl.CSRFFilter doFilter: the provided CSRF token is invalid"

Tests

create Pages

Create a bunch of geometrixx pages in the repository.

Example Usage

java -Xmx1024m -Dhostname=localhost -Dport=4502 -DcreatePages.toplevels=2 -DcreatePages.sublevels=2 -DcreatePages.subsublevels=2 -DcreatePages.concurrentUsers=3 -DcreatePages.rampup=1 -DcreatePages.thinkt=5 -DcreatePages.activate=true -DcreatePages.delete=true -jar toughday-5.6.1.jar createPages

create pages cluster

Create a bunch of geometrixx pages in a clustered repository.

Example Usage

java -Xmx1024m -Dhostname=localhost -Dport=4502 -Dhostname.slave=localhost -Dport.slave=8080 -DcreatePagesCluster.concurrentUsers=10 -DcreatePagesCluster.rampup=1 -DcreatePagesCluster.loop=10 -jar toughday-5.6.1.jar createPagesCluster

MSM

Test the MultiSiteManager (MSM) using geometrixx.

Example Usage

java -Xmx1024m -Dhostname=localhost -Dport=4502 -DCQ5MSMNewSite.count=10 -DCQ5MSMNewSite.testplan=CQ5MSMNewSite-1.0.jmx -jar toughday-5.6.1.jar CQ5MSMNewSite

DAM

Create a number of different images in the repository. This also triggers some DAM workflows
(metadata extraction, renditions etc.).

Example Usage

java -Xmx1024m -Dhostname=localhost -Dport=4502 -DuploadImage.count=10 -jar toughday-5.6.1.jar uploadImage

benchmark

Execute the createPages and uploadImages tasks and measure the time they require to run. This enables you to compare the results with those of an Amazon m1-large instance.

Example Usage

java -Xmx1024m -Dhostname=localhost -Dport=4502 -jar toughday-5.6.1.jar benchmark

tagging

Create ~20000 tags below /etc/tags/ebay.

Example Usage

java -Xmx1024m -Dhostname=localhost -Dport=4502 -jar toughday-5.6.1.jar tags

tagging 2

Add ~100000 tags to /content/geometrixx/en/jcr:content (dependent on the Tags test).

Example Usage

java -Xmx1024m -Dhostname=localhost -Dport=4502 -jar toughday-5.6.1.jar addTagsToGeometrixxPages

upload PDF

Uploads a number of PDFs. This also triggers certain DAM workflows (used for metadata extraction, etc.).

Example Usage

java -Xmx1024m -Dhostname=localhost -Dport=4502 -DuploadPdf.count=10 -jar toughday-5.6.1.jar uploadPdf

Authoring Scenario

From 5.5 onwards there are new author scenario tests which simulate all GET and POST requests (97% GET, 3% POST).

To achieve this we have defined a scenario that simulates a regular use case for authors. This includes steps including creating a page, adding some components, rating some content, activating some pages and deleting pages.

The Authoring Scenario uses the Geometrixx Demo Site:

  1. Login into CQ (http://<hostname>:<port>)
  2. Open WebSites
  3. Navigate to Geometrixx Demo Site --> English
  4. Create a new Page with a random title and the Geometrixx Content Page template
  5. Open the testPage
  6. Add a Sitemap component to the paragraph system
  7. Add a Flash component to the paragraph system
  8. Select the Movies tab inside the Content Finder
  9. Drag & Drop the item triangles.swf to the Flash Component
  10. Add a Text & Image component to the paragraph system
  11. Select the Images tab inside the Content Finder
  12. Scroll down and drag & drop the file geo_hq.jpg to the Image component
  13. Edit the Text & Image component and write following text:
    "Our triangles meet all ISO and DIN industry standards, GeoMetrixx# gives you the ability to effortlessly offer the perfect product and support."
  14. Add a Tag Cloud to the paragraph system
  15. Edit the Tag Cloud, set Tags to Display to All tags
  16. Add a Ratings component to the paragraph system
  17. Add a Comments component to the paragraph system
  18. Activate the Page with the depending assets inside the Siteadmin
  19. Open the Page
  20. Delete the Page inside the Siteadmin

Example Usage

java -Xmx1024m -Dhostname=localhost -Dport=4502 -Dauthoring.loop=2 -Dauthoring.thread=2 -jar toughday-5.6.1.jar authoring

all

Run all the tests.

Example Usage

java -Xmx1024m -Dhostname=localhost -Dport=4502 -jar toughday-5.6.1.jar all

Ce produit est distribué sous licence Creative Commons Attribution - Pas d’utilisation commerciale - Partage à l’identique 3.0 non transposé  Les publications Twitter™ et Facebook ne sont pas couvertes par les dispositions Creative Commons.

Mentions légales   |   Politique de confidentialité en ligne