Search Engine Showdown
[an error occurred while processing this directive]

Search Engines Statistics: Database Overlap
by Greg R. Notess

Data from May 5, 1999

Pie Chart Still little overlap!

Searches Used:
Total Hits:
Unique Hits:
5 small ones
267
122

Even with four Inktomi-based databases (Anzwers, Snap, MSN Web Search, and Yahoo!'s Inktomi database), there was a low degree of overlap even among those four. (Anzwers, Canada.com, and HotBot do find the same hits and Anzwers is used as an estimator of all three. Canada.com and HotBot are not considered separately here.) Each of the four Inktomi databases checked here found hits that none of the other Inktomi databases found.

See the more detailed analysis of unique hits to gain a sense of how the 63 pages found by only one search engine were distributed.

Previous Comparisons:

  • March 1999: Four search on ten search engines. 202 hits, 97 unique pages. None found by more than five search engines.
  • Jan. 1999: Four search on ten search engines. 176 hits, 83 unique pages. None found by more than six search engines.
  • August 1998: Four searches on five search engines. 103 hits, 70 unique pages. None found by all five search engines.
  • May 1998: Four searches on five search engines. 95 hits, 77 unique pages. None found by all five.
  • Feb. 1998: Four searches on five search engines. 103 hits, 62 unique pages. Three found by all five search engines.
  • October 1997: Four different searches on four search engines: 220 hits, 12 found by all four
  • September 1997 and June 1997 found no pages in common among four small searches on the four largest search engines at those times. (No charts available.)

While decisions about which Web search engine to use should not be based on size alone, this information is especially important when looking for very specific keywords, phrases, and areas of specialized interest. See also the following statistical analyses: