Introduction
Project Summary:
Use BERT model with PyTorch to do sentiment analysis. Explore various detail implementations including:
Stratified approach in splitting the dataset
Encode text into tensor
Custom BERT model with layers we want
The use of optimizer and scheduler
Training loop for fine tuning
New Concepts Involved
Transformer: an advance version of recurrent neural net. It can parallelize processing and training. You can treat it as a convolutional network, for which input size is fixed
HuggingFace: a firm that provide lots of transformer implementation
SMILE Twitter Dataset: twitter with tag showing the emotional of that tweet
tqdm: a package to check the process
Process
Exploratory Data Analysis and Preprocessing
Training/Validation Split
Loading Tokenizer and Encoding our Data
Setting up BERT Pretrained Model
Creating Data Loaders
Setting Up Optimizer and Scheduler
Defining our Performance Metrics
Creating our Training Loop
Loading and Evaluating our Model
Last updated