Keeping Public Cache Warm Indefinitely

Using LiteMage's crawler to warm your cache can lead to a huge improvement in performance by regularly re-cacheing pages that might otherwise have expired. By slightly changing your LiteMage configuration, you can set this up so this is always the case.

LiteSpeed Web Server is able to detect the difference between requests coming from the crawler and those from elsewhere allowing it to serve user requests from cache while still letting the crawler hit the backend. This results in your cache being in an always warm state with visitors never hitting the backend.

To do this, change the Default Public Cache TTL setting in the General Settings section of your LiteMage Configuration screen to an amount of time greater than your Warm Up Interval setting in the Cache Warmup section.

For example:

By setting the Default Public Cache TTL to 86400 seconds (24 hours)

and the Warm Up Interval to 43200 seconds (12 hours)

the crawler will update your publicly cached pages before their cache entries have the chance to expire.

Note: This can be done on a per-store basis by changing your Current Configuration Scope.

Note: It is possible that LiteMage will not finish crawling your site within the set Max Run Time. This can be caused by a large site or a high server load at run time, where the crawler will stop to prevent it from using up your remaining server resources. In these cases, the crawler should continue where it left off on its next run. This may cause parts of your site to not remain in cache. You can restart a crawlers queue without flushing your cache by clicking the reset link at the bottom of the System→Cache Management page in the Magento Admin Panel.