Dynamic Pages: ? and & are Red Flags

• Big Search Engines Avoid "Deep Web"

  • They worry about being sucked into a "black hole"

  • They think contents may not be interesting

  Thousands or millions of URLs

  BrightPlanet Deep Web White Paper

• Solution 1: rewrite URLs to appear static

• OLD: name?id=4&require=true

  NEW: name/id.4/require-true/index.html

• See Amazon.com for example

• Apache mod_rewrite can do this

• XQASP for ASP sites

• Pardeikes Welcome for Mac OS web servers

• Solution 2: static versions

  • Generate static versions of each page with links to dynamic versions

  • No funny characters in URLs

• Solution 3: summary pages

  • Summarize content with listings of products or information.

  • Links to database pages (robots will not follow)

  • Less optimized -- will not rank well in search results

Start | Prev | Next

Search Engine Strategies
Dallas, November 9, 2000

Avi Rappoport, Search Tools Consulting
www.searchtools.com