CDN Caching, Time-To-Live (TTL) | Scene7

Why do images expire immediately in a browser? 

Solution

CDN Caching and TTL (Time-To-Live)

With Adobe Scene7, you can set different expirations for specific images. For example, the default image that is displayed when an image is not found, is automatically configured to have a shorter expiration time.

Time-To-Live (TTL) is a mechanism to limit the lifetime of data in a network. Our Content Delivery Network (CDN) providers offer this configuration setting. When the time-to-live time has passed, a GET request for an object triggers an IMS (If-Modified-Since) request to the Scene7 origin. If our CDN provider receives a 304 response (Not-Modified) from our origin servers, the object is refreshed, along with an updated Expires header, if it has changed.

Several years ago, we had a ten-hour TTL, which was authoritative, and the Expires header had no impact on our CDN caching. This setting was changed so that when an object expired either with the CDN TTL or based on the "Expires:" header, the object was refreshed on the next "GET" request. This process also refreshed the headers. In short, we now override the CDN's TTL if the Expires header time is shorter than the configured TTL.

The above condition applies when the object is "in-cache" on Akamai. When Akamai makes an if-modified-since (IMS) request to the origin, it doesn't have any high-impact traffic, because only HTTP headers are exchanged. The object is not downloaded again. Our CDN is only checking if the object is unchanged, and refreshes the TTL of that object in cache.

The impact of this CDN configuration change was that there are now more frequent IMS requests made to the origin servers for content with a low expiration time.

CDN config changes, why would an image expire immediately in a browser?

Example:

Response Headers:
Content-Type[image/jpeg]
Last-Modified[Wed, 20 Jun 2007 21:29:20 GMT]
Etag["0b3a49e639331555ba959a9f1e332c2f"]
Expires[Mon, 13 Aug 2007 02:00:28 GMT]
Date[Mon, 13 Aug 2007 02:00:28 GMT]
Connection[keep-alive]

Several years ago, we found this object was stored in cache with an Expires header in the past:

HTTP/1.1 200 OK
Server: Apache-Coyote/1.1
Expires: Mon, 13 Aug 2007 16:53:16 GMT
Last-Modified: Thu, 19 Jul 2007 21:42:18 GMT
ETag: "ebb5a6db6c4e78a15db53a29d9d0b9d2"
Content-Type: image/jpeg
Content-Length: 14141
Date: Mon, 13 Aug 2007 06:53:15 GMT
Age: 0
X-Akamai-Purge-Seq-Num: 1336350
Connection: keep-alive

With our previous Content Delivery Network (CDN) configuration settings, this behavior was as designed. Having an object stored in cache with an Expires header in the past resulted with a downstream expires header of "now." The "Last-modified" date had not changed. As a result, an HTTP 304 response from the origin for an if-modified-since (IMS) request didn't update headers of cached objects with Akamai. When an object was cached, and expired, our CDN provider always sent an "if-modified-since" (IMS) request to origin. Because the "Last Modified" date is the same, our CDN provider received an HTTP 304 response. Then, the object cache Time-To-Live (TTL) was updated to  ten hours based on configuration, but the object and headers remained unchanged.

At that point, we updated the CDN metadata to control the update of the cached headers.

The tag that allowed us to change the cacheability headers on a 304 response from the origin server was:

<cache:header-update.allow>on</cache:header-update.allow>

By default, our CDN provider does not update the headers in a cached object with the headers received in a 304 response. As a result, once the Expires time passed, the object would remain stale in the CDN cache (and for any end user) until the origin returned a 200 response with a new object. Once the Expires time was reached, if the origin responded with a 304, the CDN server would continue to revalidate with the origin on every request. It would also serve the client-browser an object that was already expired. It caused the browser to have to revalidate on the next load.

When we enabled the cache:header-update.allow tag, that instructed the CDN servers to update the cache-directive headers in the cached response if they are different in the 304 reply. Headers considered `cache-directive' headers for this feature are: Expires, Cache-Control, Edge-Control (and implicitly Surrogate-Control).

There is also a related control over how frequently the headers are updated in cache. This control exists to prevent a situation where the CDN server is required to write headers to cache on every response because the origin server is setting a new Expires header with each response (not the case with Adobe Scene7 responses). The default setting for the frequency of header update is one minute, but this value can be changed with the metadata tag:

<cache:header-update.max-frequency>1m</cache:header-update.max-frequency>

When TTL is reached, the object is considered stale on EdgeServers. An IMS request are always sent to origin to check for last-modified time of that same object.

The metadata change we made allows for refreshing the headers along with the 304 response from the origin. This change did not result in a full download of that same object if it was not modified.

The if-modified-since request to the origin servers are only made when the content is stale. That is, after the TTL expires, the headers aren't updated that frequently.