Saturday, July 26, 2008

1 Trillion pages on the internet

Official Google Blog: We knew the web was big...
Google now indexing 1 trillion pages and counting. If you read the post above, they say it may be infinite as new pages keep getting created every minute, the crawlers will have a tough time keeping up.

In a college assignment to create a search index for an article containing 1 million words, my program took about 30 seconds to execute. 1 Trillion web pages with so many words in them, it is awesome just to imagine that.