Natural Language Processing (NLP) is a field in Machine Learning (ML) focused on teaching computers to understand and interpret text and spoken language like humans do. Recent advancements in architectures, such as the transformer architecture, have enabled NLP models to achieve near-human performance on tasks like text summarization, text classification, and entity recognition. Large language models (LLMs) are transformer-based models trained on massive amounts of unlabeled text, with parameters ranging from hundreds of millions to over a trillion. Training LLMs from scratch is extremely challenging and expensive, so a common practice is to fine-tune pre-trained LLMs.
Zero-shot learning in NLP allows pre-trained LLMs to generate responses for tasks they haven’t been explicitly trained on, without the need for fine-tuning. For example, zero-shot text classification is a task where an NLP model classifies text from unseen classes, unlike supervised classification where the model can only classify text from classes in the training data. Amazon SageMaker JumpStart is a machine learning hub that provides access to pre-trained models, algorithms, and solution templates to quickly start with ML. It recently launched support for zero-shot classification models.
Zero-shot classification is a paradigm where a model can classify new examples that belong to classes not present in the training data. This is different from fine-tuning, which requires re-training the model. A framework proposed by Yin et al. uses natural language inference (NLI) to create zero-shot classifiers. NLI poses the sequence to be classified as a premise and constructs a hypothesis from each candidate label. By evaluating the probabilities for entailment and contradiction, label probabilities are obtained.
SageMaker JumpStart simplifies the deployment and fine-tuning of pre-trained models across various ML tasks. It provides access to over 300 text models in the JumpStart model hub, enabling transfer learning and fine-tuning on custom datasets. Deploying a standalone zero-shot text classification model in SageMaker JumpStart involves accessing pre-trained models, specifying deployment configurations, and creating a SageMaker endpoint.
SageMaker JumpStart also offers solution templates for common ML use cases. The “Zero Shot Text Classification with Hugging Face” solution allows text classification without training a model for specific labels, using a pre-trained text classifier like the facebook-bart-large-mnli model. It provides a dataset for New Year’s resolutions classification and supports both real-time and batch inference.
Performing real-time inference using a zero-shot classification model involves launching the Zero Shot Text Classification with Hugging Face solution, which provisions a SageMaker real-time endpoint and an S3 bucket for storing batch transformation results. The architecture of this method is illustrated in a diagram.
Overall, SageMaker JumpStart provides a user-friendly interface and a Python SDK to deploy and utilize pre-trained zero-shot classification models for various NLP tasks.