How to Set up LiteMage Cache Warmup for a Big Site

Larger stores have special needs when it comes to cache warmup.

Your Magento store is a very large site with 17k pages. LiteMage TTL is set to 86400 (24 hours), and the warmup interval is set to 57600 (16 hours). This results in the entire store being re-cached once-a-day via the crawler, which takes 6-8 hours to complete.

This is a problem, because the website runs very slowly while it is being crawled.

For such a big site, you can adjust your settings based on your situation. How often do your products change? If they do not change much, then there is no need to have a TTL as short as 24 hours. Set your public TTL to something larger, like 604800 (1 week), and set your warm-up interval to 345600 (4 days). Every 3 days it will try to recrawl. You can also enable the delta crawler, so when something changes, the related URLs are automatically recrawled in 5 minutes.

When you flush all LiteMage cache files, the whole site warm-up will be automatically started. So it's safe to set a long TTL when you have delta crawler enabled.