Hunting with Spiders, Indexes
and Search Engines

A Collection of Search Engines

• Personal Favorites
• Selected Indexes, Search Engines and Directories
  
  • Highly Rated Engines, the Major Players
  • Multi-threaded Engines (ie, searching multiple engines at once)
  • Subject Guides
  • Special Subject Indexes and Guides
  • The Hytelnets
• Indexes to Mailing Lists
  
• Electronic Texts - books, magazines, poetry
• Learning to Use Search Engines
  
• Evaluating Search Engines
  
• Evaluating Web Site Contents
• How to Cite what you found
• How to improve your ranking in the search engines

Personal Favorites

Advanced Query mode, but the Simple Query is excellent too.

When I am casting about in a general way, Yahoo can't be beat. Their general subject categories offer a terrific way to start research.

Google is a new, fast and terrific search engine that I'm using more and more. It has an academic focus.

I'm keeping a Nettrain archive of search engines and exercises. Nettrain is a mailing list for people who teach Internet.

Indexes and Search Engines

Highly Rated Engines, the Major Players

How do you know if a search engine is a major player? Search Engine Watch attempts to keep track and offer the criteria.

• Alta Vista advanced query
  uses the + and - along with boolean AND/OR/AND NOT. Uses NEAR for proximity, quote marks for phrases. Field searching includes title:, anchor:, text:, applet:, link:, image:, URL:, host:, domain: Also uses wildcards and truncation, recognizes capitalization.
  the advanced query mode allows phrase searching and boolean nests. You can also search URLs, title fields and Usenet newsgroups. Simple Search help and see also Advanced search help

• Webcrawler uses the + and - along with boolean AND/OR/NOT. It allows phrase searching (using quotes "") but doesn't allow proximity. No wildcards, no truncation, and no reorganization of capital letters. Webcrawler help

• Excite uses the + and - along with boolean AND/OR/AND NOT. No proximity, no field searching. Can quote phrases. No wildcards, no recognition of capitalization. Excite Help

• HotBot is Lycos based.
  Allows boolean and uses the + and -. No proximity. Use quotes for phrase searches. Limited field searching. Has wildcards and truncation (use ? for a single character). Counts meta tags in relevancy. Hot bot help

• Yahoo A subject Directory AND help for yahoo

• Lycos Their advanced search is based on a fill-out form and divided into Web, News and Shopping. Lycos help

• Searching the EINet Information Galaxy. Their advanced search is form-based and allows the search to be limited to subject categories. The advanced search page contains help information and more detailed help is available.

• Northern Lights. You have to pay money for most hits!! uses the + and - and boolean AND/OR/NOT. No proximity, uses quote marks for phrases, can search fields (URL:, title:,pub:, company:, text: Allows truncation and wildcards, no recognition of capitalization, it does index meta data but assigns no special relevancy to it. Their power search is form-based.

Multi-threaded Engines

These search engines allow you to search several indexes at once. But the disadvantage is you don't get the control over the search logic that you do with individual search engines. This is because there is no standard "protocol" or construction of search strategies amongst the various engines; they all have their own separate, unique interface and search rules.

• Beaucoup contains over 600 search engines arranged by subject, type and geography. They have a large list of links to finding people. The help page was blank on 16 Oct 2002.

• Dogpile. The advanced search allows you to select which search engines you want to search. Options include any, all, phrase Search options include images, audio/mp3, message boards, multimedia. It searches yellow pages and white pages. tools and tips

• metacrawler appears to be a clone of Dogpile.

• Search.com. The advanced search includes options for subject categories. help

Subject Guides

Subject guides are a good way to formulate an approach to a research topic. By browsing, or window-shopping, one can see a variety of approaches to brand new concepts. If I'm working on a topic I've never worked with before, the subject approach is appealing because I don't yet have the vocabulary necessary for boolean and phrase searching. ooOoo it's fun too

• The Internet Public Library's Ready Reference Collection links dictionaries, thesauri, encyclopedias, and directories. It also groups resources by subject.

• Special Search Engines is a truly massive collection. Excellent source for academic research and general. Search engines are grouped by subject ranging from history to cgi and perl script archives.

• The World Wide Web Virtual Library: Subject Catalogue

• Yahoo. I include Yahoo in 2 places on this page, both as a major search engine and here amongst the subject guides.

• The Argus Clearinghouse is a wonderful for academic and non-academic browsing. Argus has closed its doors but I keep this archive linked because it's so valuable.

Special Subject Indexes and Guides

A broad and eclectic collection.

• The Scout Report is eclectic and fun. Since 1884 it's been the best source of information about the Internet available. It's a librarian's dream.

• US Government Printing Office
  Includes Federal Register, Congressional Record, Congressional Bills and more

• John December has been around a long time and the cluttered look can be discouraging. But there are valuable resources here.

• Thomas: US Legislative Information (103rd and 104th Congress)

• AskERIC is a database of educational resources.

• Google Groups is an index to Usenet

• Liszt is an index to mailing lists and a good way to search for Usenet newsgroups too. indexes the full text of web pages.

• Current Cites is an annotated monthly bibliography of selected articles, books, and electronic documents on information technology

• The Library of Congress

• Alex: A Catalogue of Electronic Texts on the Internet

• Project Gutenberg

• Electronic text center

The Hytelnets

Hytelnet is a menu-based approach to telnetting into sites around the net. They are an excellent approach to library catalogs and freenets.

• Internet Resources via HyTelnet
  Supports the boolean AND and OR and you can nest terms, ex:
  (ecology or environment) and ocean. Wow, this seems to be the last one.

Indexes to mailing lists

• The L-Soft Index to Listserv

• This collection of publicly available Mailing Lists has a very nice subject arrangement.

• The Tile.Net Reference to Internet Discussion Groups offers a search engine that first presents a list of formats: mailing lists, FTP sites and Usenet.

• The Liszt index of discussion groups

Learning to Use Search Engines

Required for my students:

• Handout on Searching (I recommend you print this)
  

• Sink or Swim: Internet Search Tools & Techniques
  A terrific resource with a good presentation of boolean search strategy and in depth comparisons of databases (including size, currency, interface and results display).

• Gillian Westera's Comparison of search engine interfaces is a great table showing the various capabilities of different search engines. She notes which search engines offer boolean, proximity, phrase, field searching, truncation (wildcards), capitalization and use of meta tags.

• The Berkeley tutorial on Finding Information on the Internet is a terrific resource.

• The Search Engine Details page at the Spider's Apprentice offers an excellent overview of the major engines, giving you the features available and a discussion of the various claims made by them.

• Search Engine Watch explains how search engines work and gives suggestions on how you can prepare your web page to be indexed.

• The Spider's Apprentice offers help on formulating search strategies and for evaluation search engines.

• Noodle Tools offers
  • bibliographies
  • strategy development
  • Choosing a search engine
  • Quick Cite to help generate an MLA-style citation for a single source (simplified for younger students).

Evaluating Search Engines

• reviews ratings and tests at Search Engine Watch

• A review of current indexing and data collection methods. This report, from Lund University in Sweden is one of the most complete I've seen.

• About paying for placement in search engines.

Evaluating Web Site Contents

• This compilation of papers on evaluating information resources contains many links to articles on how to critically analyze material.

• Evaluating Internet Based Information is from a course on the Internet and includes links to web pages along with evaluative comments on their content.

• Thinking Critically about World Wide Web Resources is a brief outline formatted guide to planning.

• A Collection of Links to Sources on Evaluating Web Resources

• Strategies, Analyzing your topic, Choosing search tools

• Evaluating Web Resources

• Preparation: Things to Know Before You Begin Searching the WWW

Improve your ranking

• Search Engines Corner, Ariadne, issue 12

• Web Developer® Library: Priming the Search Engines

• How To Use Meta Tags

• Search Engine SPAM - The Story

• Search Engine Marketing - Web Marketing Today Info Center

• PositionAgent - Monitor Web Site Search Engine Rankings On Top Search Engines

• Robots Exclusion - or how NOT to be indexed!

http://matrixmagic.com

Last Edited:
Matrix Magic Send mail to Matrix Magic