📖
Sentiment-Analysis
  • Introduction
  • Untitled
  • Unsupervised
    • Unsupervised Sentiment Analysis
    • Transfer Learning
  • PyTorch - BERT
    • What is BERT
    • Exploratory Data Analysis and Pre-processing
    • Training/Validation Split
    • Encode with Tokenizer and Prepare the dataset for Machine Learning
    • Setup the BERT Pretrained Model
    • DataLoader, Optimizer, Scheduler
    • Defining Performance Metrics
    • Set up training logics
    • Evaluate the model
Powered by GitBook
On this page
  • Load Package
  • Loading the model

Was this helpful?

  1. PyTorch - BERT

Setup the BERT Pretrained Model

BERT pre-trained model is a good starting point for us to do a lot of NLP task, and we can use its performance as Benchmark. In this session, we simply load the model

PreviousEncode with Tokenizer and Prepare the dataset for Machine LearningNextDataLoader, Optimizer, Scheduler

Last updated 4 years ago

Was this helpful?

Load Package

the package is in the library

from transformers import BertForSequenceClassification

In here, we use the one for ”Sequence Classification” because the twitter data is a sequential data: a paragraph of text

Loading the model

Model = BertForSequenceClassification.from_pretrained(
    'bert-base-uncased', # the specific model we want to use, it is matching with the tokenizer we choose
    num_labels=len(label_dict), # how many classes / labels we need
    output_attentions=False, # whether the model tells us its reasoning in making that prediction
    output_hidden_states=False
)

transformers