Text classification is an extensively used phenomenon in natural language processing which has widespread utility in the different domains. Also known as text categorization, text classification finds its usage in various tasks related to information retrieval and management. Spam detection in e-mails, opinion mining or sentiment analysis on social media data, priority e-mail sorting, intent identification from user queries in chatbots, and automated query answering mechanisms are a few examples where text categorization has proved to be highly effective. In earlier chapters, we have discussed various feature selection and dimensionality reduction methods, which are preprocessing steps before text classification. We will briefly discuss supervised learning or classification mechanisms, how a learner is designed, and then we will move on to their implementation in terms of text data. We will also discuss the different cross-validation and evaluation mechanisms to evaluate the efficiency of the classification models, briefly discussing the diagnostic procedure on learners to yield better accuracy as well.
In this chapter, we will cover the following topics:
- Classification mechanism for text analysis
- Different classification techniques in text mining, with a detailed focus on linear and probabilistic classifiers:
- Naive Bayes
- Support vector machines
- Maximum entropy
- Decision trees
- Model evaluation
- Learning curve: bias-variance tradeoff
The digital era has seen a humongous increase in data, which is unstructured and needs to be processed to extract any information out of it. Research in the field of natural language processing has paved the way towards automatic organization and classification of documents into the categories that they belong to. Document classification finds its utility in numerous applications such as spam classification, mail routing, priority inbox or mail relevance ranking, news monitoring and censoring, identification or article genre, and indexing of documents. The text classification process flow is described in the following diagram. We have discussed the preprocessing steps in Chapter 5, Text Summarization and Clustering, which involves basic data cleansing. After this step, we choose the document representation method. Features extraction and selection is performed on the cleansed data as per the document representation method chosen in the last step. We have discussed feature extraction, feature selection, and dimensionality reduction in Chapter 5, Text Summarization and Clustering. So, we will concentrate on the classification process throughout this chapter as shown in the following diagram:
