A New Generation Search Engines

The Financial Times recently published a short note: “New tools to vie with Google”, which briefly describes –from a user point of view- some of the new generation search engines. It includes new search engines like:

• http://www.ask.com/
  
• http://www.powerset.com/
  
• http://www.askmenow.com/
  
• http://www.hakia.com/
  
• http://www.digger.com/
  
• http://www.blinkx.com/ (video clips)
  
• http://www.retrivo.com/ (electronics products)
  
• http://www.kosmix.com/ (health, travel, finance, US politics)
  

All those search engines allow queries in natural language, and they claim to use NLP and semantics to “understand” the content.

In my understanding, there are two ways where NLP and Semantics can play a role:

• Interpreting the user query. This is what most new search engines claim to do.
• “Understanding” the content to be indexed. This requires that at index time, not only the individual words of the content (documents) are indexed, but indexing also considers NLP and Semantics. It is unclear to what extent those “new” search engines apply this for indexing. Yet another possibility is to launch the query against structured information (e.g. RDF).
  

I usually summarize the above two points in respectively Semantics in Access and Semantics in the Source. The latter is obviously much harder than the former, and a real challenge for the next generation search engines.

The short note of Paul Taylor also mentions several “search” engines for comparing prices of products available on the web, including

• Shopzilla
  
• Pricegrabber
  
• Shopping.com

While the note talks about those “shopping bots” for consumers, also providers (the businesses) may have interest in such tools for tracking their products at resellers’ sites and for automatically tracking their competitors’ products.