| Natural Language Processing |
Article Index for Natural Language |
Website Links For Natural Language Processing |
Information AboutNatural Language Processing |
| CATEGORIES ABOUT NATURAL LANGUAGE PROCESSING | |
| computational linguistics | |
| speech recognition | |
| natural language processingcomputational linguistics | |
| speech recognition | |
| natural language processing | |
| artificial intelligence | |
| computational linguistics | |
|
NATURAL LANGUAGE PROCESSING Early systems such as SHRDLU , working in restricted " Blocks World s" with restricted vocabularies, worked extremely well, leading researchers to excessive optimism which was soon lost when the systems were extended to more realistic situations with real-world ambiguity and complexity. Natural language understanding is sometimes referred to as an AI-complete problem, because natural language recognition seems to require extensive knowledge about the outside world and the ability to manipulate it. The definition of "understanding" is one of the major problems in natural language processing. Some examples of the problems faced by natural language understanding systems:
The word "time" alone can be interpreted as three different parts of speech, (noun in the first example, verb in 2, 3, 4, and adjective in 5). :English is particularly challenging in this regard because it has little Inflectional Morphology to distinguish between parts of speech.
THE MAJOR TASKS IN NLP
SOME PROBLEMS WHICH MAKE NLP DIFFICULT ; Speech Segmentation : In most spoken languages, the sounds representing successive letters blend into each other, so the conversion of the analog signal to discrete characters can be a very difficult process. Also, in natural speech there are hardly any pauses between successive words; the location of those boundaries usually must take into account grammatical and semantical constraints, as well as the context. ; and Thai do not have signal word boundaries either, so any significant text parsing usually requires the identification of word boundaries, which is often a non-trivial task. ; Word Sense Disambiguation : Many words have more than one meaning; we have to select the meaning which makes the most sense in context. ; for Natural Language s is Ambiguous , i.e. there are often multiple possible Parse Tree s for a given sentence. Choosing the most appropriate one usually requires Semantic and contextual information. ; Imperfect or irregular input : Foreign or regional accents and vocal impediments in speech; typing or grammatical errors, OCR errors in texts. ; Speech Acts and plans: Sentences often don't mean what they literally say; for instance a good answer to "Can you pass the salt" is to pass the salt; in most contexts "Yes" is not a good answer, although "No" is better and "I'm afraid that I can't see it" is better yet. Or again, if a class was not offered last year, "The class was not offered last year" is a better answer to the question "How many students failed the class last year?" than "None" is. STATISTICAL NLP Statistical natural language processing uses Stochastic , Probabilistic and Statistical methods to resolve some of the difficulties discussed above, especially those which arise because longer sentences are highly ambiguous when processed with realistic grammars, yielding thousands or millions of possible analyses. Methods for disambiguation often involve the use of Corpora and Markov Model s. The technology for statistical NLP comes mainly from Machine Learning and Data Mining , both of which are fields of Artificial Intelligence that involve learning from data. SEE ALSO
EXTERNAL LINKS Resources
Research and development groups
Implementations
|
|
|