How classification is used

Classifying text is used for a number of purposes:

  • Spam detection
  • Authorship attribution
  • Sentiment analysis
  • Age and gender identification
  • Determining the subject of a document
  • Language identification

Spamming is an unfortunate reality for most email users. If an email can be classified as spam, then it can be moved to a spam folder. A text message can be analyzed and certain attributes can be used to designate the email as spam. These attributes can include misspellings, lack of an appropriate email address for the recipients, and a non-standard URL.

Classification has been used to determine the authorship of documents. This has been performed on historical documents, such as The Federalist Papers and the book Primary Colors, where the authors were identified using classification techniques.

Sentiment analysis is a technique that determines the attitude of a piece of text. Movie reviews have been a popular domain for this kind of analysis, but it can be used for almost any product review. This helps companies better assess how their product is perceived. Often, a negative or positive attribute is assigned to the text. Sentiment analysis is also called opinion extraction/mining and subjectivity analysis. Consumer confidence and the performance of a stock market can be predicted from Twitter feeds and other sources.

Classification can be used to determine the age and gender of a text's author and to provide more insight into its author. Frequently, the number of pronouns, determiners, and noun phrases are used to identify the gender of a writer. Females tend to use more pronouns and males tend to use more determiners.

Determining the subject of pieces of text is useful when we need to organize a large number of documents. Search engines are very much concerned with this activity, but it has also been used simply to place documents in different categories—for example, in tag clouds. A tag cloud is a group of words that reflects the relative frequency of the occurrence of each word.

The following diagram is an example of a tag cloud generated by IBM Word Cloud Generator (http://www.softpedia.com/get/Office-tools/Other-Office-Tools/IBM-Word-Cloud-Generator.shtml), and can be found at http://upload.wikimedia.org/wikipedia/commons/9/9e/Foundation-l_word_cloud_without_headers_and_quotes.png:

The identification of the language used by a document is supported using classification techniques. This analysis is useful for many NLP problems where we need to apply specific language models to the problem.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset