Robot Spider Indexers

• Crawl from URLs and follow links

• View pages like an end-user

  • dynamic page content pre-rendered

• Channeled through server access control

• Multiple and remote servers

• Slower than local indexing

• Problem links, especially. JavaScript

Previous slide

Next slide

Back to first slide

View graphic version