Presentation by Search Tools Consulting's Avi Rappoport

Intranet Search: Best Practices

Avi Rappoport

Search Tools Consulting

Presented at the KMWorld/Intranets 2006 Conference

Background

• Avi Rappoport
  • Librarian and long-time search engine analyst
  • Fascinated by user behavior
• Search Tools Consulting
  • Editor & Publisher, SearchTools.com
  • Research, analysis, dissemination
  • Real-world consulting

Improving Search - The Big Picture

• Solid search needs some attention
  • It takes more than plugging it in and turning it on
  • Each enterprise will have surprising requirements
  • Invest resources in finding out
• Information Needs Analysis
  • Where are the problems? What's not working?
  • What is working? Don't break it!
• Content Inventory
  • Can't find what is not indexed
• User Experience Analysis
  • Search UI is the tip of the iceberg
  • Dig deeper to find functional problems

Why Not Just Use Google / Yahoo / Ask?

• Intranet search has some similarities to web-wide search
  • Find content and index it
  • Provide a simple search interface
  • Return a relevance-ranked list of matches
    
• Web-wide search specifics
  • Indexes mainly web pages
  • No control over content
  • Search Spam -- attempting to boost a site, push down competitors
• Intranet search specifics
  • Also index file shares, list archives, databases
  • Security and access control issues
  • Can make judgments about relative value of content sources
  • Provide Search Suggestions for frequent queries

Information Needs Analysis

• Start with standard requirements
  • General topics: HR, facilities, help desk and company information
  • Move towards an enterprise-wide simple search
• Special cases in your intranet - may require complex tools
  • Scientific / Technical / Academic research
• Competitive intelligence
• Sales prospects
• Where is the real pain?
  • Systems that drive users nuts
  • Obsolete tools that only a few people understand
  • Integrating new content, acquisitions
  • New employee frustration
• What works already?
  

Plan for Change

• Evaluate current search quality: be ruthless
  • Work with Intranet teams, Portals, IA
  • Check existing complaints
  • Analyze search logs
  • Send out surveys and perform interviews
• Define personas to focus plans
  • A variety of users - at least three
  • Vary by job, location, seniority, and other aspects
• Propose changes
  • Fit business goals
  • Describe current problem and offer solutions
  • Identify resource requirements

Technical Problems: Divide & Conquer

• Find and index the valuable content
• Rich indexing means better results
• Balance ease of use vs. powerful query languages
• Find ways to improve query matching
• Use relevance ranking to show the best results first
• Display navigation and context on results pages
• Show useful information in results
• Use search log analysis to discover problems (and then fix them)

Content Inventory

• Search is blamed for missing content, so find everything
  • Hidden servers
  • Tricky data sources
• Creating a Content Inventory is the only way to keep track
  • Work with existing Intranet teams (Portal, IA, Taxonomy)
  • Identify valuable content and metadata
    • Consider entity extraction
• Keep track of ROT: Redundant, Obsolete and Trivial items
• Store language and character set information
• Security issues
  • Privacy: try not to index people's desktops
• Access control and authorization
• Define or apply policies and note them in the inventory

Rich Indexing For Better Results

• Index the full text: every word of each indexed item
  • Don't worry about stopwords or stemming here
  • For database records, index contents of all text fields
  • Include context of each word, including meta tags, field names, and position in text
• Store extensive information about each item
     (web page, office file, database record, etc)
• Location (URL, file path, record ID)
• Title and other explicit metadata
• Size, date, file format
• Security and access control
• Character set and language
• Leverage taxonomy and IA labels - enormously valuable
  • Taxonomy categories allow users to search by category

About Modification Dates

• Content dating provides valuable context
• Reliable modification dates are as rare as hen's teeth
  • Misconfigured servers send ludicrous dates, in the distant past or future
• File modification date may not be accurate
• Dynamic sites send the date of page assembly, not content changes
• Encourage accurate dating
  • Encourage server webmasters to fix date service
  • Can use metadata tags instead of server defaults
• Content inventory can track item changes
  • Use a checksum to see if anything has changed
  • Search can show a note, "not changed since Jan 16, 2002"

Metadata and Entity Extraction

• Item metadata - meta tags or properties, creator or editor
  • Like a folksonomy
  • More trustworthy than on the Web - no spam
  • But not entirely - unless convinced, people won't tag at all
  • Vocabulary mismatches
• Entity extraction
  • Partly based on Natural Language Processing
  • Identify people, places and things within documents and text items
  • Improves relevance ranking significantly
  • Will need some editorial oversight for confusing vocabulary
  • Store as metadata in a repository- do not perform the extraction at index time

Indexing Multimedia

• Images, audio and video search
• Look at the business case, define its value to the enterprise
• Tricky process
  • Index the file or field name and text near the links to it
  • Index any transcript or related text
  • Use taxonomy tools to categorize multimedia items
• Extract text from image or sound
  • Subtitles, OCR for text in graphics, speech-to-text systems
  • Will need editorial oversight
  • Store as metadata for search and reading

The Search Process

Query Processing and Retrieval

• Search field interface
  • Expect short queries - all studies say the average is 1-3 words
  • Check search field length - too short, you get shorter queries
  • Advanced search is useful for complex tasks
• Query expansion - valuable process, ignored if done right
  • Lightweight pluralization
  • Synonyms and acronym conversion
  • Spellchecking - now expected, and rightly so
• Matching queries to items: recall vs. precision
• Is more better? Sometimes, on smaller sites or ecommerce
• Consider character set and language
• Be clear, let users know how it works
• Never default to phrase-matching -- it's very frustrating for users

Relevance Ranking

• Ranking matched items by likelihood that they will be useful
  • The most common method, the TF:IDF algorithm
    • Counts matching words in the item
    • More weight for words rarely used in the index
  • Other algorithms: vectors, statistical and linguistic analysis
  • Heuristics: more weight for phrase matches, matches in title or metadata tag
• Learn how the search engine's relevance ranking works
  • Test and try until it makes sense
  • Decode the relevance numbers
  • Consider how to show users what you've learned
• Adjust weighting to bring up more valuable content
  • Some directories are more equal than others

Elements of Search Results

• Keep it simple (most of the time)
  • Complex graphical interfaces may work for topical research
  • Show context, navigation links on result page
  • Never set the default to search within results: no one understands it
• Spelling suggestions
• Search suggestions
• Landing pages for frequent queries
• Useful for linking to web applications on separate servers
• Results items
  • Balance simplicity against additional information
  • Transparency - why did this item get retrieved?
    • Show matched terms with surrounding text

Results Page Example

Taxonomy Integration

• Taxonomies improve search
  • Standardized labels and vocabulary
  • Show what subjects the items cover
• Search supplements taxonomy hierarchies
  • An alternate method of navigation, avoid drill-down
• Crosses all taxonomy categories
• Handles user vocabularies
• Linking them together
  • Store taxonomy information with documents at index time
  • Allow search within taxonomy categories
  • Show taxonomy labels in results

Security & Access Control

• Be careful -- indexing can expose private materials
• Work with security auditors
• Simple access control:
  • Show all results, let security system ask for passwords
• Hit-level access control
• Check with security system before displaying search results
• Always current, never shows disallowed results, can be slow
• Highest-level security
• Protect server hardware and software
• Limit and log access to the search interface
• Encrypt transmissions during indexing and serving results

Faceted Metadata in Results

• Faceted Metadata Search and Browse
  • Facets = aspects or attributes of an item
  • Not just topical, also locations, dates, file types, content types
• Facets enrich the results
  • Show context - multiple ways to consider the item
  • Provide alternate navigation, drill-down
• Leverage the "scent of information"
• Requires structured data
  • Can make use of entity extraction and other metadata
  • Appropriate for research rather than enterprise-wide intranet search

Faceted Metadata Example

Dealing With Search Failure

• User - Search mismatch
• User typed in uppercase and the content is lowercase or vice versa
• Query has only partial matches to items in the index
• Spelling or typing mistakes
• Query words are synonyms of those used in the content
• Users misunderstand search scope
  • Assume the search field is like Google and goes everywhere
  • Type domain names in the search field
  • Search in a subsection, think it's the whole intranet
• Create a useful no-matches page
  • Provide context and suggestions
• Server errors
• Demand 99.99% uptime and graceful restarts
• Expect notification of all errors - users shouldn't bring down a server

Example - Good No-Matches Page

Reality Check - Search Log Analysis

• Metrics
• Number of items indexed, indexing errors
• Queries per day, week, month
• Server issues
• Search Logs
  • Raw material for text mining
  • Not the same as web logs
  • Search-specific textual information, not just data points
• Measuring search success
  • Number of queries, click-throughs, no-matches, next-pages
  • Top queries and no-matches
  • Tracking "sessions" by IP and timestamp, watch user behavior

pagebreak

Using Analytics to Tune Search

• Change search engine behavior
  • Remove case sensitivity from query processing
  • Consider matching items that contain any search word, not all search words
• Add content that users want
  • Common synonyms and acronyms as metadata
• Pages on new topics
• Use Search Suggestions to handle special cases
  • Popular searches
    • Top 1% of query phrases account for 15% - 20% of searches
  • Human judgment works well
  • Requires continuing maintenance and resources

Personalization in Search

• Simple tools work great
  • Saving searches as bookmarks or links
  • Search history
• Opt-in alerting services
  • Start with existing query and send notification of new items that fit
  • Some false matches, especially if quality requirement is low
  • Users are fairly tolerant
• Strict personal or role-based retrieval and ranking is not worth it
  • Individual activity is not statistically significant
  • Users have multiple roles at the same time
    • A search for "colon" might mean the body part or the character
    • Even scientists need to know about the holiday schedule
  • Everyone's information needs change

Complex Research Tasks

• More complex research
• Technical and scientific research
• Business intelligence
• Sales prospect research
• Regulatory compliance
• Particularly important for new hires
• Intranet-wide search gets them to the right place
• Specialized search engines still needed
• Allow simple query syntax, like the intranet search
• Add more tools
• Conceptual search, using statistical and linguistic analysis
• Rich results: faceted metadata, clustering, visualization

Measure Search Progress

• Go back to the Information Needs Analysis
  • What has been solved, or partly solved? What hasn't?
  • Are persona tasks easier?
  • Complaints down and compliments up?
• Other metrics
  • Increase in valuable content indexed
    • Possible decrease in overall size if ROT is gone
  • Increase in use of the search engine and intranet
    • Search use may go down if intranet navigation improves
  • Greater number of users
  • Fewer calls to help desks
  • Less duplication, more re-use of edited materials
  • Survey user satisfaction

Conclusion

• Plenty of work to get the basics done
• Keep track of changing information needs
  • Update personas
  • Consider special search if the ROI justifies it
• Index all the good stuff you can get
  • Work with content creators
  • Encourage logical metadata - title tags/properties at least
• Analyze the logs
  • Encourage new content
• Add Search Suggestions
• Invest in maintenance