There are a lot of ways to warmup the cache, but each method does the same. It does request URLs of your page like LiteSpeed crawler does it. The question is if your provider will check this and disallows each method that crawls your page. The best way would be to kick this provider off. He can't offer LiteSpeed services, but disallows crawler. Stupid!
You can use "Xenu". A local Windows application that is meant for checking broken links and works like crawler.
http://home.snafu.de/tilman/xenulink.html
You can use "Xenu". A local Windows application that is meant for checking broken links and works like crawler.
http://home.snafu.de/tilman/xenulink.html
For WordPress I used Wp Super Preload.
For other sites, not limited to WP, from same author there is Preload ByCron.
Still, it must be configured and tested.
@serpent_driver
Great idea with broken links plugin, thanks. Never crossed my mind.
For other sites, not limited to WP, from same author there is Preload ByCron.
Still, it must be configured and tested.
@serpent_driver
Great idea with broken links plugin, thanks. Never crossed my mind.
@Germont
Preload ByCron crawler script is good, but he overlooks something in curl requests, that causes too much load and traffic and as result of this, requests take too much time. There are also some other deficits....
Preload ByCron crawler script is good, but he overlooks something in curl requests, that causes too much load and traffic and as result of this, requests take too much time. There are also some other deficits....
In 2013 curl already had commands I use to make requests much faster.
Possible, but this script (no plugin) is part of an application and uses sources from this application. I have to extract this sources from application to make a standalone version. It is already on my to do list, but with low priority. Ask my again in some weeks....
Possible, but this script (no plugin) is part of an application and uses sources from this application. I have to extract this sources from application to make a standalone version. It is already on my to do list, but with low priority. Ask my again in some weeks....
Which feature do you need? Free Netflix access while crawling, free entry to all soccer (football) games in premier league in your country, a dozen virgins? 
I have integrated all features that are possible with cURL. Only cURL limits what is possible and impossible. Witout cURL no HighSpeed crawling!
I have integrated all features that are possible with cURL. Only cURL limits what is possible and impossible. Witout cURL no HighSpeed crawling!
Now I recalculated and it seems in WordPress I can preload even 10k urls / hour at full load with PHP8 and Memcached.
But I ignored the VPS I manage is still on MySQL 5.7, and it seems MySQL 8 has 2x more speed.
What version do you have for PHP/ MySQL?
Do you also use Object Caching?
That partially explains the difference.
But I ignored the VPS I manage is still on MySQL 5.7, and it seems MySQL 8 has 2x more speed.
What version do you have for PHP/ MySQL?
Do you also use Object Caching?
That partially explains the difference.
Memcached, Object cache or any other caching doesn't matter for warmup crawling. I noted no difference if it is used or not, because I am not shure if these kind of cache is available in command line interface (CLI).
On this shared hosting I have PHP 7.4 and MariaDB, but it seems also not matter which versions are used. The difference in speed doesn't depend on any installed software for PHP and MySQL. It depends on how URLs are requested and how powerful the server is.
On this shared hosting I have PHP 7.4 and MariaDB, but it seems also not matter which versions are used. The difference in speed doesn't depend on any installed software for PHP and MySQL. It depends on how URLs are requested and how powerful the server is.