I’ve created a new page and placed it on my homepage hoping it will get indexed within 2 days but it didn’t.
MSN indexes faster than Google and Yahoo. The webpage got indexed in MSN in 6 days because even though the first crawl was on the 16th of March. Google crawled the page on the 16th as well and a little bit before MSN but the page is still not in Google’s index. Yahoo crawled the page on the 17th but it still doesn’t show up in index too.
Since i’ve got that page up, Google has crawled it just once, MSN four times and Yahoo twice.
Anyway, we’ll see when the page is going to show up in the other search engines.
Below are the logs for GoogleBot, MSNBot, Yahoo!Slurp and AskBot.
66.249.72.10 – – [16/Mar/2007:00:10:48 -0500] “GET /learners/driving-instructor-training/ HTTP/1.1” 200 13399 “-” “Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)”
65.55.209.205 – – [16/Mar/2007:15:58:23 -0500] “GET /learners/driving-instructor-training/ HTTP/1.0” 200 13347 “-” “msnbot/1.0 (+http://search.msn.com/msnbot.htm)”
65.55.213.14 – – [18/Mar/2007:20:10:30 -0500] “GET /learners/driving-instructor-training/ HTTP/1.0” 200 13347 “-” “msnbot-media/1.0 (+http://search.msn.com/msnbot.htm)”
65.54.188.139 – – [20/Mar/2007:19:45:52 -0500] “GET /learners/driving-instructor-training/ HTTP/1.0” 200 13347 “-” “msnbot/1.0 (+http://search.msn.com/msnbot.htm)”
65.55.235.216 – – [20/Mar/2007:21:23:33 -0500] “GET /learners/driving-instructor-training/ HTTP/1.0” 200 13347 “-” “msnbot-media/1.0 (+http://search.msn.com/msnbot.htm)”
74.6.85.231 – – [17/Mar/2007:04:06:28 -0500] “GET /learners/driving-instructor-training/ HTTP/1.0” 200 13347 “-” “Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp)”
74.6.70.72 – – [21/Mar/2007:13:18:02 -0500] “GET /learners/driving-instructor-training/ HTTP/1.0” 200 13347 “-” “Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp)”
65.214.44.157 – – [18/Mar/2007:10:34:18 -0500] “GET /learners/driving-instructor-training/ HTTP/1.0” 200 13347 “-” “Mozilla/2.0 (compatible; Ask Jeeves/Teoma; +http://about.ask.com/en/docs/about/webmasters.shtml)”