Getting Pages Into Indexes: Traditional Solutions

• Robot Spiders Through Links

  • Delay while locating new & updated pages

  • Indexes navigation text or requires special rendering

  • Servers often don't send correct file modification date

  • Additional load on server

• File System Paths

• Proprietory Gateway APIs for Indexing

  • Avoid indexing navigation text

  • Reduce effort

  • Synchronize publication and search indexing

  • BUT: each search engine writes special code -- no one can afford to do them all

Previous | Next | Contents

>Open Source Search and Content Management
OSCOM, September 25, 2002
Avi Rappoport SearchTools.com