Web Tutorial

Search Engines vs. Directories

There are two fundamentally different types of search services available on the Web, and it's important to understand the difference between them.

Search engines are actually full-text indexes of Web pages. They're similar to telephone white pages, which contain nothing more than lots of names with phone numbers. You won't find a description of a person or what kind of house they live in in the white pages, but they're organized in a way that makes it very easy to look up somebody named "Smith" or "Veerhoven."

Search engine indexes are compiled by software "robots" which voraciously suck millions of pages into the index every day. When you search an index, you're trying to coax it to find a good match between the keywords you type in and all of the words contained in the index. In essence, you're relying on a computer to essentially do simple pattern-matching between your query words and the words in the index. AltaVista, HotBot, and Infoseek are examples of indexes.

Directories, on the other hand, are collections of links to Web sites compiled by people, not software robots. Directories are similar to telephone yellow pages, because they are organized by category or topic, and often contain more information than bare-bones white pages listings.

Directories have the advantage over search engines in that they can be very precise in how they categorize pages. Many directories annotate their links with descriptions or comments, so you can get an idea of what a site is about before clicking through. About.com, LookSmart, and Yahoo! are examples of directories.

Metasearch engines enable you to enter a single search phrase and search a number of web indexes simultaneously. These sites are especially valuable if your search term is relatively obscure and you receive relatively few hits from a single index search engine or dirtectory. The most useful are Dogpile, MetaCrawler, and Altaseek.

Regional search engines: regional and local search engines and country specific Web directories located around the world.

Which Is Better?

The key advantage of an index is size - indexes typically have information on millions and millions of Web pages. This size, unfortunately, can also be a problem for common subjects. There's often no way to determine the quality of a link in a results list you get from a search engine's index.

The advantage of a directory is that the links will almost always be of much higher quality, but the scope of the results necessarily will be much smaller than that returned by an index, so you might miss something a search engine might have found. Another key advantage of directories is that they are organized in a hierarchical format, which lets you "drill down" through subject trees, "browsing" for links in much the same way you browse for books through stacks in a library.

So the answer is that neither is "better" - in general, a search engine is best when you want lots of results, and a directory is better when you want focused results.

Learn To Be A Power Searcher

If you want to get really good at searching, it pays to spend time with a good tutorial that provides lots of examples of different kinds of searches, and the best tools to use to get the results you want. Here are two excellent searching tutorials, and a glossary to help you learn the meaning of search terms.

Finding Information on the Internet: A Tutorial
From the Teaching Library at UC Berkeley, these outstanding guides offer beginning, intermediate, and advanced sections on using the Internet's resources to find information. Handouts distributed for the library's drop-in classes are also available.

Bright Planet's Complete Planet
This is the largest compilation of searchable databases and engines on the Internet -- 40,000 listings and growing placed in a 7,000 category directory structure. Their great tutorial: Guide to Effective Searching of the Internet, shows you practical steps in harnessing the Internet to improve the quality of your search results.

Glossary of Search Engine Terms
This extensive glossary covers the gamut of specialized language used by search engines. In addition to providing hyperlinked cross references, there are also links to all of the major search sites on the Web.

Intelligent Search Agents and Bots
Search agents and bots are intelligent, autonomous searching tools that scour the Web on your behalf.