Introduction

Project Summary:

Use BERT model with PyTorch to do sentiment analysis. Explore various detail implementations including:

  1. Stratified approach in splitting the dataset

  2. Encode text into tensor

  3. Custom BERT model with layers we want

  4. The use of optimizer and scheduler

  5. Training loop for fine tuning

New Concepts Involved

  1. Transformer: an advance version of recurrent neural net. It can parallelize processing and training. You can treat it as a convolutional network, for which input size is fixed

  2. HuggingFace: a firm that provide lots of transformer implementation

  3. SMILE Twitter Dataset: twitter with tag showing the emotional of that tweet

  4. tqdm: a package to check the process

Process

  1. Exploratory Data Analysis and Preprocessing

  2. Training/Validation Split

  3. Loading Tokenizer and Encoding our Data

  4. Setting up BERT Pretrained Model

  5. Creating Data Loaders

  6. Setting Up Optimizer and Scheduler

  7. Defining our Performance Metrics

  8. Creating our Training Loop

  9. Loading and Evaluating our Model

Last updated