Search Tools Product Report

ht://Dig

Product Information
Mailing List Archive

Price: Free, open source under the GNU General Public License.
Platform: Solaris, HP/UX, IRIX, SunOS, Linux, Mac OS X, Mac OS 9 (from Tenon.)

Potential Security Flaw: 3.1.0b2 through 3.1.5 and 3.2.0b3 users should upgrade.

Features

• Free and open-source, written in C++.
  
• Extremely helpful user group.
  
• Designed for source-level modification and customization.
  
• ConfigDig: a template-based HTML front end for easy search administration from any browser. This allows remote configuration by search admins who are not expert at Unix command-line interfaces.
  
• Handles multiple sites and over 100,000 pages.
  
• Index spider is quite robust and handles error conditions gracefully.
  
• Metadata indexing is configurable, easy to add Dublin Core DC tags.
  
• Can index PDF, MS Word, PowerPoint (see tips.)
  
• Many options for indexing and searching non-exact matches, including stemming, soundex, synonyms, metaphone, and fuzzy matching.
  
• Supports keyword and Boolean expression searching.
• Note: version 3.2 will haa phrase matching, 3.1 only does AND or OR or Boolean search.

Articles & Reviews

• Comparison of Free Search Engines SearchTools.com: April 2006 by Yiling Chen
  
  Concentrates on indexing, query processing, retrieval and relevance. Covers Alkaline, Fluid Dynamics, ht://Dig, Juggernautsearch, mnoGoSearch, Perlfect, SWISH-E, Webinator, and Webglimpse. All of these search engines are are free, mainly on Unix, and many are open source software.


• The Open Road: Using ht://Dig UnixReview: April 2002 by Joe "Zonker" Brockmeier
  
  Part 1 is a short but helpful discussion of how the indexing and search work, formatting results, scheduling and configuration. Part 2 talks about tuning the search engine for speed and efficiency.


• Comparing Open Source Indexers Infomotions Musings: May 29, 2001 by Eric Lease Morgan
  
  Describes the history and features of eight open-source search engines, freeWAIS-sf (aging code and hard to install, but good for searching email and public domain etexts); Harvest (powerful gathering features for frequently-changing data stores, good with structured documents); ht://Dig (tricky to configure, no phrase searching, automatic stemming and match word highlighting); Isearch (weak documentation and support, easy to install, dated interface, Z39.50 support); MPS Information Server (zippy indexing of both text and structured data, Z39.50 support, Perl API, limited documentation); SWISH-E (simple to install engine, CGIs in Perl and PHP still beta, good for HTML pages, recognizes new META tags, sorts results by field; WebGlimpse (easy to install and configure, requires commercial version for customized output); Yaz/Zebra (mainly Z39.50, no Perl API, mainly a toolkit to index and respond to distributed client queries). Article also points out that chaotic information is less than helpful and encourages organization, structure and vocabulary control.


• I love it when a plan comes together PalmPower Magazine: March 2001 by David Gewirtz
  
  Rambling but cheerful description of setting up a search engine for ZATZ web sites using ht://Dig, indexing only the appropriate articles and not the alternate forms or contents pages. Some digressions into robots.txt, Linux and PHP.


• Search Engines: The Hunt Is On Network Computing Magazine: October 16, 2000 by Avi Rappoport
  
  In-depth discussion of search engines for e-commerce and other web sites covers features and future trends, software vs. services, database vs. text searching, natural-language searching, and open-source search engines covering ht://Dig and mnoGoSearch.

Examples

• Alfred University - small college
• Morrissey Solo - An active fan site for English solo musician Morrissey.
• Functional Materials for Energy Technology - German research institution devoted to developing and optimizing materials and highly-efficient systems for thermal insulation.

Page Modified: 2007-03-23