Jump to content
To block spammers, this forum has suspended new user registration ×
Comet Forums
To block spammers, this forum has suspended new user registration

how to download offline pages without being stopped by the website Bot?

Sam Rag

Recommended Posts


I started just recently to use your software and I guess it is great. I wanted to offline download some pages from this website, that uses a very strong bot.
See examples of the pages





it started to work fine, and downloaded the pages as should be but after 40 links it stopped and the rest of the pages did not download. I understand we need to respect that website and not to overload it with download requests, but is there a way for example to batch add 2000 links and have an option in your software that says download one page ever 2 or 3 seconds and not consecutively? Or at least does some temporization as if a human being is downloading the pages, and not the software! Is there a slow but sure way to do this? Pardon me if my question sounds naiive, but I am new to your software. I hope someone can help.

Thanks a lot.



Link to comment
Share on other sites

Please sign in to comment

You will be able to leave a comment after signing in

Sign In Now
  • Create New...