Text Data Management and Analysis: A Practical Introduction to Information Retrieval and Text Mining

Recent years have seen a dramatic growth of natural language text data, including web pages, news articles, scientific literature, emails, enterprise documents, and social media such as blog articles, forum posts, product reviews, and tweets. This has led to an increasing demand for powerful software tools to help people analyze and manage vast amounts of text data effectively and efficiently. Unlike data generated by a computer system or sensors, text data are usually generated directly by humans, and are accompanied by semantically rich content. As such, text data are especially valuable for discovering knowledge about human opinions and preferences, in addition to many other kinds of knowledge that we encode in text. In contrast to structured data, which conform to well-defined schemas (thus are relatively easy for computers to handle), text has less explicit structure, requiring computer processing toward understanding of the content encoded in text. The current technology of natural language processing has not yet reached a point to enable a computer to precisely understand natural language text, but a wide range of statistical and heuristic approaches to analysis and management of text data have been developed over the past few decades. They are usually very robust and can be applied to analyze and manage text data in any natural language, and about any topic.

This book provides a systematic introduction to all these approaches, with an emphasis on covering the most useful knowledge and skills required to build a variety of practically useful text information systems. The focus is on text mining applications that can help users analyze patterns in text data to extract and reveal useful knowledge. Information retrieval systems, including search engines and recommender systems, are also covered as supporting technology for text mining applications. The book covers the major concepts, techniques, and ideas in text data mining and information retrieval from a practical viewpoint, and includes many hands-on exercises designed with a companion software toolkit (i.e., MeTA) to help readers learn how to apply techniques of text mining and information retrieval to real-world text data and how to experiment with and improve some of the algorithms for interesting application tasks. The book can be used as a textbook for a computer science undergraduate course or a reference book for practitioners working on relevant problems in analyzing and managing text data.

Table of Contents

PART I. OVERVIEW AND BACKGROUND
Chapter 1. Introduction
Chapter 2. Background
Chapter 3. Text Data Understanding
Chapter 4. Meta: A Unified Toolkit For Text Data Management And Analysis

PART II. TEXT DATA ACCESS
Chapter 5. Overview Of Text Data Access
Chapter 6. Retrieval Models
Chapter 7. Feedback
Chapter 8. Search Engine Implementation
Chapter 9. Search Engine Evaluation
Chapter 10. Web Search
Chapter 11. Recommender Systems

PART III. TEXT DATA ANALYSIS
Chapter 12. Overview Of Text Data Analysis
Chapter 13. Word Association Mining
Chapter 14. Text Clustering
Chapter 15. Text Categorization
Chapter 16. Text Summarization
Chapter 17. Topic Analysis
Chapter 18. Opinion Mining And Sentiment Analysis
Chapter 19. Joint Analysis Of Text And Structured Data

PART IV. UNIFIED TEXT DATA MANAGEMENT ANALYSIS SYSTEM
Chapter 20. Toward A Unified System For Text Managment And Analysis

Appendix A. Bayesian Statistics
Appendix B. Expectation-Maximization
Appendix Kl-Divergence And Dirichlet Prior Smoothing