December 03, 1999
National Research Council
Design of a Better Question Answering System
A better search-engine would allow you to ask a natural language question and would return an answer instead of 10,000 web pages. In this talk I will review our current design for question answering and summarize the design of all the other systems that were presented at TREC-8 in Gaithersburg, MD. Our system, like almost all the other systems, does 'passage retrieval' (find a short passage likely to contain the answer) and then scans for answer types that match the question (for a Who question look for a person). The most obvious deficiencies in our current design are speed and the accuracy of "answer-type" identification. Other passage retrieval systems can do a search in 5-25 seconds while ours takes minutes and other named-entity system components have greater than 90% accuracy while ours is around 50%.
I will finish the talk with an outline of the design of a better, more robust question answering system.
This work was done with Chris Lankester from the University of Ottawa.