This article focuses on the latest optimizations in the AEM dispatcher and how to best leverage those. The AEM dispatcher is a caching reverse proxy server designed for use with Adobe Experience Manager. It can be installed and runs as a module within existing web server software. At the time of writing this article, the dispatcher module is supported on Apache HTTP Server, Microsoft IIS and iPlanet.
At the most basic level, the AEM dispatcher is a reverse proxy which works by performing caching, cache flushing and cache invalidation.
See the related links for more details on dispatcher:
Here are some ways to optimize the dispatcher cache:
As of Dispatcher version 4.1.11, /enableTTL 1 can be set in the .any file configuration. This setting makes the dispatcher respect cache expirations set in the HTTP Cache-Control response header. In other words, the dispatcher will function similar to a CDN where primary form of cache invalidation occurs when files expire. Once you implement this and start sending Cache-Control: max-age=... for all responses from AEM, then you can safely disable your dispatcher flush agents in the publish instances.
After disabling flush agents on the publish instances then you may still want to be able to flush the dispatcher cache. In that case, you can use ACS Commons - Dispatcher Flush UI. This tool is installed on the author instance. It gives users a UI where they can perform manual cache flush requests.
I. Steps to enable TTL ("Time to Live" or expiration) style invalidations:
II. Disable dispatcher flush agents on the publish instances:
The dispatcher will now use the Cache-Control header to control invalidation of the cache files. Since that is the case, then dispatcher flushing from the publish instances is no longer required.
III. Allow manual dispatcher flush requests from the author instance:
Now that flush agents are disabled, you would rely entirely on the Cache-Control header to control when content is refreshed on the dispatcher. You can still allow users to issue manual flushes of the dispatcher cache:
To optimize dispatcher flush requests, all dispatcher flush agents should have a feature called refetching flush enabled.
To enable re-fetching dispatcher flush, do the following:
Note that the package installed above is just a basic example. To customize and optimize re-fetching flush you can modify the list of URIs that it sends. The code is open source and can be found here. The code adds a list of URIs to the request body as parameters telling dispatcher which paths to re-fetch. You can add more paths per your application requirements to optimize your site's caching capabilities.
Normally a dispatcher flush works by deleting files:
Due to the fact that files are deleted in step 2, the next time a user requests a file like /content/foo.html or /content/foo.json, while the file is being "re-fetched" then subsequent requests for the same file would also be sent to the publish instances until the file is cached. For slow responses or heavy traffic pages such as home pages this can cause flooding of the publish instance tier.
To solve this issue, enable a feature of the dispatcher called re-fetching. This feature allows you to send a list of URIs that the dispatcher should proactively "re-fetch" and replace instead deleting.
See 22:41-27:05 in this presentation recording for a demo of how it works and how to configure it.