Jin Liang, Software Engineer at Google, gives a brief overview on Googlebot and web hosting. Find out more about robots.txt, Google’s crawl rate, and how HTTP serving has evolved.
Change Googlebot crawl rate →
Watch all the recorded talks from WMConf MTV ’19 →
Subscribe to the Webmasters Channel →
source
