How Fast Do Search Engines Index Fresh Content

Yesterday was a unique day in the blogosphere. Following the crackdown by Digg staff, the infamous hex key - 09 f9 11 02 9d 74 e3 5b d8 41 56 c5 63 56 88 c0 - was written all over the Internet from blogs to news portals to social websites.

Legal issues apart, this mutiny at Digg gives us a great opportunity to compare the indexing behaviour of search engines and how frequently search bots scroll the web in pursuit of fresh content.

We executed a search for 09 f9 11 02 9d 74 e3 5b d8 41 56 c5 63 56 88 c0 (without quotes) on all major web and blogs search engines and here are the results.



Microsoft owned Live search has just 18k pages containing the HD DVD key while Google has indexed over half a million documents containing that number.



In case of Blog Search Engines, Bloglines (owned by Ask.com) found 6160 blogs and RSS feeds that mentioned the HD DVD key while Technorati suggested the number as 2200. Google Blog Search results were ~5.5k.