Keeping Public Cache Warm Indefinitely

Using LiteMage's crawler to warm your cache can lead to a huge improvement in performance by regularly re-cacheing pages that might otherwise have expired. By slightly changing your LiteMage configuration, you can set this up so this is always the case.

LiteSpeed Web Server is able to detect the difference between requests coming from the crawler and those from elsewhere allowing it to serve user requests from cache while still letting the crawler hit the backend. This results in your cache being in an always warm state with visitors never hitting the backend.

To do this, change the Default Public Cache TTL setting in the General Settings section of your LiteMage Configuration screen to an amount of time greater than your Warm Up Interval setting in the Cache Warmup section.

For example:

By setting the Default Public Cache TTL to 86400 seconds (24 hours)

and the Warm Up Interval to 43200 seconds (12 hours)

the crawler will update your publicly cached pages before their cache entries have the chance to expire.

Note: This can be done on a per-store basis by changing your Current Configuration Scope.

Note: LiteMage will not crawl your site all at once. In order to prevent the crawler from using all of your server's resources and allow flexibility, the crawler will be run once every 10 minutes (600 seconds). You can set the Maximum Run Time for a crawler run to any value less than 600 seconds, the default duration being 5 minutes (300 seconds). When the crawler reaches this time limit, it stops until it's next run and then resumes where it left off. If your Default Public Cache TTL setting is too low, parts of your site will not remain in cache.

Note: You can restart a crawlers queue without flushing your cache by clicking the reset link at the bottom of the System→Cache Management page in the Magento Admin Panel.

Note: Users can also set their own Custom Defined Url list in case they wish to refresh certain URLs more or less frequently than their crawler queues.

Note: If, after updating a page, a user wants to refresh the cached copy of this page, they can use the following command in a cron job to refresh the cached copy programmatically:

curl -A "litemage_walker" https://your_url

Refreshing the cache is not the same as invalidating it. When the cached copy is invalidated, the next user to hit the invalidated page will have to wait longer for the page to load as there is no copy to serve from cache. When the cache is refreshed, the cached copy is overwritten with the current version of the page.

The difference between the LiteMage Purge command and the curl command listed above is that the Purge command invalidates the cache for the current page, while the curl command refreshes the cached copy of the given URL.