Applied Natural Language Processing in the Enterprise: Teaching Machines to Read, Write, and Understand
- Length: 336 pages
- Edition: 1
- Language: English
- Publisher: O'Reilly Media
- Publication Date: 2021-06-08
- ISBN-10: 149206257X
- ISBN-13: 9781492062578
- Sales Rank: #1384813 (See Top 100 Books)
NLP has exploded in popularity over the last few years. But while Google, Facebook, OpenAI, and others continue to release larger language models, many teams still struggle with building NLP applications that live up to the hype. This hands-on guide helps you get up to speed on the latest and most promising trends in NLP.
With a basic understanding of machine learning and some Python experience, you’ll learn how to build, train, and deploy models for real-world applications in your organization. Authors Ankur Patel and Ajay Uppili Arasanipalai guide you through the process using code and examples that highlightthe best practices in modern NLP.
- Use state-of-the-art NLP models such as BERT and GPT-3 to solve NLP tasks such as named entity recognition, text classification, semantic search, and reading comprehension
- Train NLP models with performance comparable or superior to that of out-of-the-box systems
- Learn about Transformer architecture and modern tricks like transfer learning that have taken the NLP world by storm
- Become familiar with the tools of the trade, including spaCy, Hugging Face, and fast.ai
- Build core parts of the NLP pipeline–including tokenizers, embeddings, and language models–from scratch using Python and PyTorch
- Take your models out of Jupyter notebooks and learn how to deploy, monitor, and maintain them in production
Preface What Is Natural Language Processing? Why Should I Read This Book? What Do I Need to Know Already? What Is This Book All About? How Is This Book Organized? Conventions Used in This Book Using Code Examples O’Reilly Online Learning How to Contact Us Acknowledgments Ajay Ankur I. Scratching the Surface 1. Introduction to NLP What Is NLP? Popular Applications History Inflection Points A Final Word Basic NLP Defining NLP Tasks Set Up the Programming Environment spaCy, fast.ai, and Hugging Face spaCy fast.ai Hugging Face Perform NLP Tasks Using spaCy Download pretrained language models Tokenization Part-of-speech tagging Dependency parsing Chunking Lemmatization Named entity recognition Named entity linking Conclusion 2. Transformers and Transfer Learning Training with fastai Using the fastai Library ULMFiT for Transfer Learning Fine-Tuning a Language Model on IMDb Training a Text Classifier Inference with Hugging Face Loading Models Generating Predictions Conclusion 3. NLP Tasks and Applications Pretrained Language Models Transfer Learning and Fine-Tuning NLP Tasks Natural Language Dataset Explore the AG Dataset NLP Task #1: Named Entity Recognition Perform Inference Using the Original spaCy Model Custom NER Annotate via Prodigy: NER Train the Custom NER Model Using spaCy Custom NER Model Versus Original NER Model NLP Task #2: Text Classification Annotate via Prodigy: Text Classification Train Text Classification Models Using spaCy Conclusion II. The Cogs in the Machine 4. Tokenization A Minimal Tokenizer Hugging Face Tokenizers Subword Tokenization Building Your Own Tokenizer Conclusion 5. Embeddings: How Machines “Understand” Words Understanding Versus Reading Text Word Vectors Word2Vec A new hope The English language strikes back The Return of the Character-Level Models Embeddings in the Age of Transfer Learning Embeddings in Practice Preprocessing Model Training Validation Embedding Things That Aren’t Words A sonnet in the MIDI protocol Making Vectorized Music Some General Tips for Making Custom Embeddings Conclusion 6. Recurrent Neural Networks and Other Sequence Models Recurrent Neural Networks RNNs in PyTorch from Scratch Bidirectional RNN Sequence to Sequence Using RNNs Long Short-Term Memory Gated Recurrent Units Conclusion 7. Transformers Building a Transformer from Scratch Attention Mechanisms Dot Product Attention Scaled Dot Product Attention Multi-Head Self-Attention Adaptive Attention Span Persistent Memory/All-Attention Product-Key Memory Transformers for Computer Vision Conclusion 8. BERTology: Putting It All Together ImageNet The Power of Pretrained Models The Path to NLP’s ImageNet Moment Pretrained Word Embeddings The Limitations of One-Hot Encoding Word2Vec GloVe fastText Context-Aware Pretrained Word Embeddings Sequential Models Sequential Data and the Importance of Sequential Models RNNs Vanilla RNNs LSTM Networks GRUs Attention Mechanisms Transformers Transformer-XL NLP’s ImageNet Moment Universal Language Model Fine-Tuning ELMo BERT BERTology GPT-1, GPT-2, GPT-3 Conclusion III. Outside the Wall 9. Tools of the Trade Deep Learning Frameworks PyTorch TensorFlow Jax Julia Visualization and Experiment Tracking TensorBoard Weights & Biases Neptune Comet MLflow AutoML H2O.ai Dataiku DataRobot ML Infrastructure and Compute Paperspace FloydHub Google Colab Kaggle Kernels Lambda GPU Cloud Edge/On-Device Inference ONNX Core ML Edge Accelerators Cloud Inference and Machine Learning as a Service AWS Microsoft Azure Google Cloud Platform Continuous Integration and Delivery Conclusion 10. Visualization Our First Streamlit App Build the Streamlit App Deploy the Streamlit App Explore the Streamlit Web App Build and Deploy a Streamlit App for Custom NER Build and Deploy a Streamlit App for Text Classification on AG News Dataset Build and Deploy a Streamlit App for Text Classification on Custom Text Conclusion 11. Productionization Data Scientists, Engineers, and Analysts Prototyping, Deployment, and Maintenance Notebooks and Scripts Databricks: Your Unified Data Analytics Platform Support for Big Data Support for Multiple Programming Languages Support for ML Frameworks Support for Model Repository, Access Control, Data Lineage, and Versioning Databricks Setup Set Up Access to S3 Bucket Set Up Libraries Create Cluster Create Notebook Enable Init Script and Restart Cluster Run Speed Test: Inference on NER Using spaCy Machine Learning Jobs Production Pipeline Notebook Scheduled Machine Learning Jobs Event-Driven Machine Learning Pipeline MLflow Log and Register Model MLflow Model Serving Alternatives to Databricks Amazon SageMaker Saturn Cloud Conclusion 12. Conclusion Ten Final Lessons Lesson 1: Start with Simple Approaches First Lesson 2: Leverage the Community Lesson 3: Do Not Create from Scratch, When Possible Lesson 4: Intuition and Experience Trounces Theory Lesson 5: Fight Decision Fatigue Lesson 6: Data Is King Lesson 7: Lean on Humans Lesson 8: Pair Yourself with Really Great Engineers Lesson 9: Ensemble Lesson 10: Have Fun Final Word A. Scaling Multi-GPU Training Distributed Training What Makes Deep Training Fast? B. CUDA Threads and Thread Blocks Writing CUDA Kernels CUDA in Practice Index
Donate to keep this site alive
How to download source code?
1. Go to: https://www.oreilly.com/
2. Search the book title: Applied Natural Language Processing in the Enterprise: Teaching Machines to Read, Write, and Understand
, sometime you may not get the results, please search the main title
3. Click the book title in the search results
3. Publisher resources
section, click Download Example Code
.
1. Disable the AdBlock plugin. Otherwise, you may not get any links.
2. Solve the CAPTCHA.
3. Click download link.
4. Lead to download server to download.