Search Engine Marketing
A slightly jaundiced view
Avi Rappoport
Search Tools Consulting
(SLIS graduate)
About Avi Rappoport
- Medieval Studies Major
- BMUG (Berkeley Mac User Group) 1986-92
- MLIS (Masters in Library and Information Studies), 1987-8
- Information retrieval, user vocabulary & thesaurus
- Worked in software development
- EndNote
- Metrowerks CodeWarrior
- StarNine Mail & WebSTAR web server
- Search Tools Consulting, since 1998
- Web site and intranet search engines
- SearchTools.com
- Research, analysis, dissemination
- Real-world consulting
handout
Elements of Search Engine Marketing
- Search Engines
- Find pages
- Respond to user demand
- Provide information
- Avoid spam
- Web Sites
- Products or Information
- Optimizing for Search Results Ranking
- Search Advertising
- Conversion
handout
Search Processes
- Search engine robot discovers web pages
- Indexer stores words for lookup, and page info
- User types a search query
- Search engine retrieves pages containing words that match
- Search engine sorts the pages by relevance
- Relevance is tricky -- like truth, beauty and pornography
- Always depends on the situation
- Especially hard for very short queries
- Browser displays the search results page (SRP)
Robots and Gathering
- Robots / Spiders / Crawlers
- Follow links to find web pages
- Ad-hoc movement , not pre-directed
- Obey "robots.txt" and "robots" meta tags
- Prefer reasonable HTML links
- Simple <a href links are best
- Problems with Java, JavaScript and complex links
- Serious problems with form interfaces
- Create a list of links to all "pages" in database
- Revisiting known pages
- Check for changes
- Problems with date tracking
placeholder
Storing Text in Index
- Every word on the page
- Includes image "ALT" tags
- File names and URL path folder names
- "Anchor text" - text of links to the page
- It has to be text!
- Pictures of text will not do, indexers are "blind"
- Hyphenated and formatted text (PDF, Word) can be weird
- Diacritics and non-Roman text: Unicode is best
- Stores every word individually
- inverted index - alphabetical by word
- Link to the source document
- Context info: tag name, position in page
placeholder
note: about stopwords
Storing Document Information
- Whole documents provide context for search
- Document properties
- URL
- Title tag or property
- MIME type or extension
- Size
- Date
- Internal metadata
- External metadata (category, department)
- Full text (sometimes)
- Show match term in context
- Display PDF and other formats in HTML
placeholder
Search and Retrieval
- User types a query in a search box
- Search engine processes the query
- "Tokenize" - break into words (hard in CJK)
- Recognize operators (AND, OR, NOT, +, =, "")
- Queries the index for each term
- Finds all documents which contain at least one of the words
- Combines retrieved document lists
- For any word match: adds all lists together
- For all words match: only keeps documents with all words
- For phrase match: only keeps documents with words in order
- For not match: removes documents with the unwanted words
- More complex operations available too
placeholder
Search and Retrieval Diagram
placeholder
Relevance Ranking
- Relevance is in the eye of the beholder
- Classic example: search for bank
- Relevance algorithms
- Number of matching terms
- Extra weight for unusual terms
- Text of incoming links
- Matching phrases in title, H1
- thousands of "ties" in relevance algorithms
- Search engines just have to punt
- Show a variety of results
- Show search terms in context
- Try to ignore search spam
placeholder
SRP: Search Results Page
- Search "header"
- Logo
- Zones
- Search field with query
- Number of hits
- Links to advanced & help, other options
- Advertising
- Above, next to, or interspersed with results
- Results List
- Results items in relevance order
- Numbered or unnumbered
- May group results from a single site
placeholder
talk about groups here
Search Results Items
- Results items
- Page Title (as a link)
- Optional description of page or site
- Text from page, with match terms marked
- URL
- Size, but rarely date
- Links to options - cache, similar items, etc.
- Advertising
- Very similar to results
- Must be very clearly marked
placeholder
advertising note re FTC ruling
Site Optimization for Search
- Content
- Make sure each page is worth finding
- Focus on a single topic, repeat important terms
- Quality of writing helps
- Titles are key
- Main chance to identify your page
- Keep it descriptive, clear and relatively short
- URLs
- Use descriptive terms in folder and file names
- Internal links - at least one link to each page
- Incoming Links
- Links from other sites, directories (Yahoo, Open Directory)
- High Page Rank is nice
placeholder
Robots.txt and Robots tag
- Simple communication with indexing robots
- "Disallow" only
- Voluntary
- Robots.txt
- File sits in top level of site
- Says not to index certain directories or files
- Can be robot-specific or apply to all
- Robots meta tags
- In page header
- NOINDEX, NOFOLLOW
- Applies to all robots
placeholder
SEO Spam and Scams
- DON'T DO THESE THINGS
- Bad for customers
- Search engines will punish the site
- Fakeout text, repeating keywords
- White on white
- Tiny font, behind a picture
- Gateway pages, client sniffing
- Show the search engine one thing, humans a different thing
- Link farms
- Bunches of sites all pointing towards the site
- Not true popularity
placeholder
Search Engine Advertising
- Key words and phrases
- Do research and define the most valuable search terms
- Short is good but longer is more qualified
- Look at competitors
- Metrics
- Click Through Rate (CTR) = clicks per impression
- Cost Per Click
- Conversions
- People buying stuff!
- Or doing what you want: registering, clicking, reading....
- Web logging
- Follow tracks through site
- Learn from successful sales
placeholder
SEM Agency Scams
- "Number One Ranking for your keywords"
- No way to guarantee this for competitive search terms
- Obscure search term rankings are pretty easy
- Link farms and other spam is often discovered
- Any "Secret Hole" will be patched soon
- "Top Ad spot for pennies"
- Impossible for competitive terms
- Click fraud
- Clicking on ads to make the numbers go up
- Automated tool or outsourced workers
- Can cost a lot of money
- Competitive nastiness
placeholder
Summary of Search Engine Marketing
- Deliver value
- Offer something worth finding
- Optimization
- Text, user-oriented vocabulary
- Focus pages on specific topics
- Make good title tags
- Advertising
- Be modest - start with specialized keywords
- Match landing pages to search terms
- For More Information
- searchenginewatch.com
- enquiro.com/marketing-monitor
placeholder