openbmb
/

MiniCPM-Embedding

Feature Extraction

sentence-transformers

Eval Results (legacy)

Model card Files Files and versions

跨语言评测

#5

by zhilinw6 - opened Oct 12, 2024

请问如何做的MKQA En-Zh_CN (Recall@20) NeuCLIR22 (NDCG@10) NeuCLIR23 (NDCG@10)跨语言评测，有统一的官方评测方法或榜单吗（如C-MTEB），或者您这边有没有开源脚本来做相关跨语言retrieval评测呢

OpenBMB org Oct 12, 2024

我们使用openmatch进行测试 https://github.com/Kaguya-19/OpenMatch

•

edited Oct 12, 2024

感谢感谢！还请问下

数据用的是这几个么 https://github.com/apple/ml-mkqa/blob/main/dataset/mkqa.jsonl.gz，https://huggingface.co/datasets/mteb/neuclir-2022或者https://huggingface.co/datasets/neuclir/neuclir1
整体模型训练时长大概是多少呢

OpenBMB org Oct 12, 2024

mkqa: https://github.com/FlagOpen/FlagEmbedding/tree/master/C_MTEB/MKQA
neuclir-22 和 neuclir-23 来自https://huggingface.co/datasets/mteb/neuclir-2022，https://huggingface.co/datasets/mteb/neuclir-2023
训练时长约5天

Kaguya-19 changed discussion status to closed Oct 12, 2024

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment