DDoS attack or bot overloading AEM with traffic

CQ5.x, AEM 6.x, AEM Dispatcher

Cause

Potential causes could be any of the following:

• Denial of Service attack
• Search bot or scraping bot hitting expensive URLs
• Extra traffic spike due to popular article, press release, etc.

Resolution

To debug such an issue, it is best to have proper logging enabled at the dispatcher level:

1. Enable logging of the X-Forwarded-For header:

In the Apache HTTP Server's access_log, add this to the httpd.conf file in the dispatcher servers:

LogFormat "%{X-Forwarded-For}i %h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
LogFormat "%h %l %u %t \"%r\" %>s %b" common
LogFormat "%{Referer}i -> %U" referer
LogFormat "%{User-agent}i" agent

In Microsoft IIS add X-Forwarded-For in the "Advanced Logging" configuration.

That would log the correct end-user IP.

Use the output showing the IP and User-Agent to analyze whether it is a malicious attack and block the offending IPs if it makes sense.

2. Set dispatcher.log's log level to debug:

Apache:
In the httpd configuration files there should be a section for the dispatcher.  Set DispatcherLogLevel to 3:

DispatcherLogLevel 3

IIS:

Modify the dis_iis.ini and set the loglevel to 3:

loglevel=3

3. Review this documentation and webinar on dispatcher caching. Take steps to improve dispatcher caching, that helps avoid outages caused by traffic spikes:

• https://docs.adobe.com/docs/en/dispatcher.html#How%20Dispatcher%20performs%20Caching
• https://github.com/cqsupport/webinar-dispatchercache
• https://docs.adobe.com/docs/en/dispatcher/disp-install.html#Apache Web Server - Configure Apache Web Server for Dispatcher