Sentimental Analyzer V3

Sentimental Analyzer V3 is a multi-task Natural Language Processing (NLP) model designed to analyze English text through two complementary tasks:

5-class sentiment classification
28-class multi-label emotion classification

The model uses microsoft/deberta-v3-base as its transformer backbone and includes a frozen production calibration pipeline for emotion predictions.

Model Overview

Property	Value
Model	Sentimental Analyzer V3
Backbone	`microsoft/deberta-v3-base`
Framework	PyTorch
Maximum sequence length	256
Sentiment classes	5
Emotion classes	28
Sentiment task	Single-label classification
Emotion task	Multi-label classification
Production calibration	`HYBRID_ISOTONIC_GLOBAL`
Calibrated emotion classes	11
Raw emotion classes	17
Global emotion threshold	0.35
Fallback strategy	`highest_calibrated_probability`

What the Model Does

Given an English text or comment, the model produces:

One sentiment class from five possible sentiment categories.
One or more emotion labels from 28 possible emotion categories, with a fallback selection when no class crosses the production threshold.
A probability distribution over the five sentiment classes.
A probability for each of the 28 emotion classes.

The model is designed for text analytics, exploratory NLP, research, educational demonstrations, and applications that analyze user feedback.

Sentiment Classification

The model predicts one of five sentiment categories:

ID	Sentiment
0	Negative
1	Neutral
2	Positive
3	Very Negative
4	Very Positive

Sentiment classification is a single-label classification task. The sentiment probabilities are normalized using softmax.

Emotion Classification

The model performs multi-label emotion classification.

ID	Emotion
0	admiration
1	amusement
2	anger
3	annoyance
4	approval
5	caring
6	confusion
7	curiosity
8	desire
9	disappointment
10	disapproval
11	disgust
12	embarrassment
13	excitement
14	fear
15	gratitude
16	grief
17	joy
18	love
19	nervousness
20	neutral
21	optimism
22	pride
23	realization
24	relief
25	remorse
26	sadness
27	surprise

Multiple emotions can be associated with the same text.

Architecture

The model uses microsoft/deberta-v3-base as the transformer encoder.

                         Input Text
                             |
                             v
                  DeBERTa-v3-base Encoder
                             |
                             v
                     Attention Pooling
                             |
                             v
                   Shared Representation
                        /           \
                       /             \
                      v               v
             Sentiment Head      Emotion Head
                      |               |
                      v               v
                5 Sentiments      28 Emotions

Attention Pooling

The sequence-level hidden representations produced by DeBERTa are aggregated using learned attention pooling. Padding tokens are masked before the attention softmax so that padded positions do not contribute to the pooled representation.

Shared Representation

The pooled representation is passed through:

Linear projection
GELU activation
Layer normalization
Dropout

The resulting representation is shared by the two task-specific prediction heads.

Sentiment Head

The sentiment head produces five logits. Softmax is applied during inference to obtain the sentiment probability distribution.

Emotion Head

The emotion head produces 28 logits. Sigmoid probabilities are used because emotion classification is multi-label.

Production Calibration

The production emotion pipeline uses the frozen strategy:

HYBRID_ISOTONIC_GLOBAL

Raw Emotion Logits
        |
        v
Sigmoid Probabilities
        |
        v
Class-wise Isotonic Calibration
        |
        v
Global Threshold = 0.35
        |
        v
Final Emotion Labels

Calibration Configuration

Calibration strategy: HYBRID_ISOTONIC_GLOBAL
Calibrated emotion classes: 11
Raw emotion classes: 17
Minimum calibration support: 10
Global threshold: 0.35
Fallback: highest_calibrated_probability

The following 11 emotion classes use isotonic calibration:

admiration
anger
annoyance
approval
disappointment
disapproval
disgust
excitement
joy
love
neutral

The following 17 rare classes retain their raw sigmoid probabilities:

amusement
caring
confusion
curiosity
desire
embarrassment
fear
gratitude
grief
nervousness
optimism
pride
realization
relief
remorse
sadness
surprise

Calibration parameters and the threshold were selected using validation data. The final test set was not used for calibration or threshold selection.

Evaluation

The final V3 checkpoint was evaluated on an untouched real-world test split containing 253 examples.

Sentiment Results

Metric	Score
Accuracy	0.8024
Macro F1	0.6499
Weighted F1	0.7864

Emotion Results

Metric	Score
Micro F1	0.5770
Macro F1	0.3026
Weighted F1	0.5968
Macro Precision	0.2791
Macro Recall	0.4198

These results correspond to the frozen V3 checkpoint and production calibration configuration.

Evaluation Interpretation

The sentiment task shows stronger performance than the 28-class emotion task.

The emotion task is more challenging because:

It is multi-label.
Some emotions are substantially rarer than others.
Emotion boundaries can be subjective.
Multiple emotions can coexist in the same text.
Some classes have limited calibration support.

Macro-level emotion metrics are particularly sensitive to performance on rare classes.

Dataset and Labeling Limitation

The real-world adaptation dataset used during development contains LLM/pseudo-labeled examples rather than a fully human-annotated gold-standard dataset.

Therefore, the reported real-world adaptation results should be interpreted as evaluation against the project's labeling methodology rather than definitive human-gold benchmark performance.

The model may perform differently on text that differs substantially from the adaptation data, including different domains, writing styles, topics, vocabulary, slang, cultural contexts, levels of context, or forms of sarcasm and implicit meaning.

Intended Use

Sentimental Analyzer V3 is intended for:

Sentiment analysis of English comments and short text.
Emotion analysis of user feedback.
NLP research and experimentation.
Educational demonstrations.
Exploratory analysis of text datasets.
Aggregate analysis of customer or user feedback.
Integration into applications through an inference API.

The model can be used as an inference component behind a web application or API.

Limitations

This model should not be considered a perfect representation of human emotion.

Important limitations include:

Emotion classification can be subjective.
Some emotion classes are substantially rarer than others.
Multi-label emotion annotation can be difficult to validate consistently.
Sarcasm and implicit meaning may be challenging.
Domain shift may reduce performance.
Rare emotion classes have limited calibration support.
The real-world adaptation labels are not fully human-gold.
The model has primarily been developed and evaluated for English text.
Model predictions are statistical estimates and should not be treated as objective facts about a person's feelings.
Predictions should not be interpreted as psychological, medical, or clinical assessments.

Example

Input

I absolutely love this product! It is amazing.

Example Production Output

{
  "sentiment": "Very Positive",
  "sentiment_confidence": 0.987,
  "emotions": [
    "joy"
  ]
}

The exact probability values depend on the frozen checkpoint and production calibration artifacts.

Model Artifacts

Sentimental-Analyzer-V3/
│
├── README.md
├── config.json
│
├── model/
│   └── best_v3_model.pt
│
├── calibration/
│   └── v3_isotonic_calibrators.pkl
│
└── labels/
    ├── sentiment_labels.json
    └── emotion_labels.json

Artifact Descriptions

model/best_v3_model.pt: Frozen V3 PyTorch checkpoint containing the trained model parameters.
calibration/v3_isotonic_calibrators.pkl: Frozen emotion calibration parameters used by the production inference pipeline.
labels/sentiment_labels.json: Mapping between sentiment class IDs and sentiment labels.
labels/emotion_labels.json: Mapping between emotion class IDs and emotion labels.
config.json: Frozen production configuration containing model, class, calibration, decision, selection, and checkpoint integrity information.

Checkpoint Integrity

The published checkpoint corresponds to the frozen V3 production candidate.

SHA-256:

4d3f5a97d352460d41fa884fd337b2de6f58549139135cad1b4b79beefcb2078

The checkpoint was verified against this SHA-256 hash before publication.

Frozen Production Configuration

Version:
V3_FINAL_CANDIDATE

Model:
microsoft/deberta-v3-base

Maximum sequence length:
256

Sentiment classes:
5

Emotion classes:
28

Calibration strategy:
HYBRID_ISOTONIC_GLOBAL

Calibrated emotion classes:
11

Raw emotion classes:
17

Minimum calibration support:
10

Global emotion threshold:
0.35

Fallback:
highest_calibrated_probability

Selection dataset:
validation

Test used for selection:
false

Threshold source:
validation

Calibration source:
validation

No additional training or calibration is performed during production inference.

Inference Behavior

Input text
    |
    v
Tokenization
    |
    v
DeBERTa-v3-base
    |
    v
Shared representation
    |
    +-------------------------+
    |                         |
    v                         v
Sentiment logits        Emotion logits
    |                         |
    v                         v
Softmax                  Sigmoid
    |                         |
    v                         v
Sentiment label        Frozen calibration
                              |
                              v
                       Threshold = 0.35
                              |
                              v
                       Emotion labels

The model is loaded in evaluation mode and its parameters are frozen during inference.

API Integration

The model can be exposed through a FastAPI service.

A production API can provide:

POST /predict

Example request:

{
  "text": "I absolutely love this product!"
}

Example response:

{
  "sentiment": "Very Positive",
  "sentiment_confidence": 0.98,
  "sentiment_probabilities": {
    "Negative": 0.00,
    "Neutral": 0.00,
    "Positive": 0.01,
    "Very Negative": 0.00,
    "Very Positive": 0.98
  },
  "emotions": [
    "joy"
  ],
  "emotion_probabilities": {
    "joy": 0.76
  }
}

The API layer can be deployed separately from this model repository so that API users do not need to download the model checkpoint directly.

Reproducibility

The V3 production candidate is frozen.

The following components are fixed:

Model checkpoint
Model architecture
Tokenizer configuration
Maximum sequence length
Sentiment label mapping
Emotion label mapping
Emotion ordering
Emotion calibration artifacts
Global emotion threshold
Fallback strategy
Production configuration

The checkpoint can be verified using the published SHA-256 hash.

Responsible Use

This model produces statistical predictions from textual patterns learned during training and adaptation.

Its outputs should not be presented as certain facts about:

A person's actual emotional state
A person's intentions
A person's personality
A person's psychological condition
A person's medical or clinical status

Applications using this model should provide appropriate context and should avoid presenting model predictions as definitive judgments.

Project Status

V3 production candidate — frozen.

The model, calibration configuration, and production inference behavior have been verified before publication.

Training and calibration are not performed during production inference.

License

MIT

Downloads last month: 20