NLPForUA/mdeberta-v3-ua-squad-reader

This model is a Ukrainian extractive Question Answering model based on microsoft/mdeberta-v3-base.

It was trained and published as an educational artifact for Lab 3: Open-domain Question Answering from the Odesа Polytechnic National University ML assignments repository.

In the laboratory assignment, the model is used as the Reader component of a two-stage ODQA system:

1. a Retriever selects relevant text passages;
2. this model extracts an answer span from a selected passage.

Training

The model was fine-tuned on the Ukrainian SQuAD dataset FIdo-AI/ua-squad.

The dataset was split by context rather than by individual question-answer pairs to avoid placing questions based on the same source context into different splits.

The resulting split contained:

Split	QA examples
Train	11,080
Validation	1,339
Test	1,440

Training configuration:

• Base model: microsoft/mdeberta-v3-base
• Epochs: 3
• Learning rate: 3e-5
• Train batch size: 16
• Evaluation batch size: 16
• Weight decay: 0.01
• Maximum sequence length: 384
• Document stride during training preprocessing: 128
• Model selection criterion: validation loss

Validation loss by epoch:

Epoch	Validation loss
1	1.1212
2	1.0247
3	1.1246

The checkpoint from epoch 2 was selected as the final model because it had the lowest validation loss.

Evaluation

The model was evaluated on the held-out test split using SQuAD 2.0 metrics.

Metric	Score
Exact Match	58.26
F1	71.99
HasAns Exact Match	57.43
HasAns F1	74.71
NoAns F1	61.49

Usage

from transformers import pipeline

model_id = "NLPForUA/mdeberta-v3-ua-squad-reader"

qa = pipeline(
    "question-answering",
    model=model_id,
    tokenizer=model_id,
)

result = qa(
    question="Яке місто є столицею України?",
    context="Київ є столицею України.",
    handle_impossible_answer=True,
)

print(result)

Intended use

The model is primarily published for educational use. It can also be used as a general Ukrainian extractive QA model.

Limitations

The model performs extractive Question Answering: it selects an answer span from the provided context rather than generating an answer from its own knowledge.

The reported results correspond to the dataset split and evaluation procedure used in the laboratory assignment and are not intended as a state-of-the-art benchmark claim.