How to Setup Crawler?

[jamesalan]

Hello,

My wordpress based online gaming website is running on Litespeed-enabled cloud server. I am using Translatepress to make the website multilingual. I could successfully improve the speed of my website's default (English) language version with the help of the Litespeed plugin. But the problem is that other language versions of URLs load very slowly.

I have more than 75k URLs on my website. Whenever I post a new game, there are 10 other language versions of post URLs created automatically.

My first question is what tweak do I have to follow to improve the speed of other language version URLs?

Next, This is the settings of the crawler, is this fine or will I have to change the values to something else based on my server and the number of URLs I have on my website?



Next, why I am seeing a high of number of URLs on crawler miss and a low number of URLs on crawler hit?


Please advise.

 

[serpent_driver]

My first question is what tweak do I have to follow to improve the speed of other language version URLs?

There is nothing that has to be tweaked. If URLs changes if a different language is selected then the sitemap needs to contain all URLs for each language to warmup the cache for all languages.

I have more than 75k URLs on my website.

75k URLs means

75000 (URLs) * 8 (cache vary) * 10 (languages) = 6000000 or 6 millions

URLs to crawl for cache warmup. In theory it would be possible to crawl such a huge number of URLs, but not in practice. You need a different warmup strategy, but the built-in crawler does not allow this. For this you need a 3rd party solution.

Next, why I am seeing a high of number of URLs on crawler miss and a low number of URLs on crawler hit?

blue icon doesn't mean "miss". It just means that a URL is successfully crawled. The green icon means that the URL is already cached and the cache TTL is updated.