sujayC66/text_summarization_512_length_1_4000
Viewer • Updated • 3.97k • 70 • 1
How to use Sivakkanth/youtube_comments_summarizer with Transformers:
# Use a pipeline as a high-level helper
# Warning: Pipeline type "summarization" is no longer supported in transformers v5.
# You must load the model directly (see below) or downgrade to v4.x with:
# 'pip install "transformers<5.0.0'
from transformers import pipeline
pipe = pipeline("summarization", model="Sivakkanth/youtube_comments_summarizer") # Load model directly
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
tokenizer = AutoTokenizer.from_pretrained("Sivakkanth/youtube_comments_summarizer")
model = AutoModelForSeq2SeqLM.from_pretrained("Sivakkanth/youtube_comments_summarizer")This model is fine-tuned to summarize YouTube comments into a concise summary.
It is based on T5 and can be used directly with the Hugging Face transformers pipeline.
from transformers import pipeline
# Load the summarization pipeline from Hugging Face
model_id = "Sivakkanth/youtube_comments_summarizer"
summarizer = pipeline("summarization", model=model_id, tokenizer=model_id)
# Sample YouTube comment text
comments_text = """
This is a really interesting video about natural language processing.
I learned a lot about different techniques for text summarization.
The presenter explained everything clearly and the examples were helpful.
I would recommend this video to anyone interested in NLP.
"""
# Generate summary
result = summarizer(
comments_text,
max_length=128,
min_length=30,
do_sample=False
)
print("Original Text:")
print(comments_text)
print("\nGenerated Summary:")
print(result[0]['summary_text'])
Input:
This is a really interesting video about natural language processing.
I learned a lot about different techniques for text summarization.
The presenter explained everything clearly and the examples were helpful.
I would recommend this video to anyone interested in NLP.
Output (example):
This video about NLP was very informative and clearly explained, with helpful examples.
## Eval Results
Evaluation on a held-out YouTube comments test set:
- **ROUGE-1:** 0.5676652376831697
- **ROUGE-2:** 0.3758989832045812
- **ROUGE-L:** 0.4824726190654699
Unable to build the model tree, the base model loops to the model itself. Learn more.