Taming bots not to consume extensive amount of hosting resources has been an ongoing task for us. After the successful launch of the new Anti-bot AI system, which has already blоcked more than 1 billion hits only from malicious bots, we’d like to shed some more light on another measure in that area – the crawl-delay setting. Read on to find out what it is, why you might want to consider it and why we no longer apply a default crawl-delay setting on our servers.