LiteMage Crawler Warm-up Status

In your Magento Admin Panel, at the bottom of the System → Cache Management page, is a section where you can monitor and manage the crawler's status.

Each crawler queue is listed as its own row in the table, sorted ascending based on priority. Included with each crawler listing is:

• Base URL - The URL from which the crawler starts crawling this queue.
• Custom List Path - This is the location of the list of paths the crawler will crawl for this queue. This can be viewed by clicking on the “Details” link for each queue.
• Run Interval (in seconds) - The time interval at which the crawler will run this queue. Refer again to our first example above, Run Interval has a value of 23400 seconds or 6.5 hours meaning that the crawler will crawl this queue every 6.5 hours.
• Store Public TTL (in seconds) - The time interval at which the public cache will be invalidated for this site. The first site above is set so the public cache is purged every 28800 seconds or every 8 hours.
• Generated - The time and date when the crawler queue was last generated.
• Finished - The time and date when the crawler last finished crawling this queue. This only appears if the crawler finished crawling this queue.
• Environment - A listing of all environment varies set for this queue. In the first row above, the varies listed are “-” which represents the default currency, USD, and GBP.
• Current Vary - The vary the crawler is currently crawling, this will remain as the last vary from your Environment list if the crawler finishes this queue.
• Current Position - The current position of the crawler in this queue.
• List Size - The size of the crawler's queue before varies are included.
• Last Query Time - The last time and date a page was crawled in this queue.
• Total Queried - The current total number of pages crawled in this queue including varies. When the crawler completes the queue, this should be equal to the number of varies times List Size. In the first row above, we have 3 varies (default, USD, and GBP). Because the first queue was completely crawled, 534 pages were crawled. 3 varies times 178 pages equals 534 total pages.

A crawler queue can be reset by clicking the corresponding “Reset” link in this section. This will remove values for: Custom List Path, Generated, Finished, List Size, and Last Query Time which will be repopulated when the crawler regenerates the list and crawls it. This will also set Current Position and Total Queried back to zero, starting the queue from the beginning the next time the crawler crawls it.