Strategy 3: Search Engines for Large Searches

Search Engines attempt to find and index as many sites as possible. Search features vary greatly, as does the actual scope, size, and accuracy of the databases. Use these for

• Unique Keywords
• Combinations of unique keywords
• Field searching and limiting
• Pages buried deep in a Web site

What's Not Included

Search engines include millions of pages in their databases, but none of them come close to indexing the entire Web, much less the entire Internet. Here is a list of some of what is missing:

• The content in sites requiring a log in
• CGI output such as data requested by a form
• Intranets; pages not linked from anywhere else
• Commercial resources with domain limitations
• Sites that use a robots.txt file to keep files and/or directories off limits
• Non-Web resources