• "Near real-time" search: a new way to search the current in-memory segment before the index has been written to disk.
• FieldCache - takes advantage of the fact that most segments of the index are static, only processes the parts that change, save on time and memory. Also improved efficiency.
• NumericField and NumericRangeQuery - (previously called TrieRange). This improves the Lucene number indexing, and is faster for searching numbers, geo-locations, and dates, faster for sorting, and hugely faster for range searching.
• Faster wildcard and prefix searching, and a reverse string filter to enable leading wildcards
• Lucene Local (Contrib / Spatial) - can limit queries based on geographic location
• Faster searching over multiple segments
• Better and faster term vector highlighting of match terms in context on results page.
• New Query Parser framework, supports additional syntaxes
• Improvements to Payloads (metadata about index terms)
• TokenStream strong typing options
• Improved transaction processing
• Better Chinese, Arabic, and Persian support

There are significant changes in version 2.9 - described in the changes.txt file or the web site (change log). A very few items are not backward compatible and several classes are deprecated. All applications should re-compile against the new Lucene 2.9 JAR 2.and test carefully.

Version 3.0 will no longer support Java 1.4 and deprecated classes. As soon as Lucene 2.9 is released,

Carrot2 3.1.0 will come out with bug fixes Solr 1.4 will use Lucene 2.9 JAR, coming soon, few weeks they hope

Note: this content extracted painfully by Avi from the Lucene site/wiki/JIRA/mailing list archive, and clarified by Grant Ingersoll's webcast sponsored by Lucid Imagination. I will be happy to fix mistakes and clarify confusion, just comment or send a message and I'll fix it.