MKLLM-7B-Translate

MKLLM-7B is an open-source Large Language Model for the Macedonian language. The model is built on top of the amazing Mistral-7B-v0.1 model by continued pretraining on a mix of Macedonian and English text. A corpus of around 300M tokens, repeated in 2 epochs, was used for the training and even though this might be considered small compared to other similar projects, the resulting model is very capable in understanding and processing the Macedonian language.

This is the translate-tuned version of MKLLM-7B. It was trained by taking MKLLM-7B and then performing QLoRA instruction training with axolotl by using the alpaca format.

We tested the model on the Flores dataset against some translation specific models, the much larger Llama3-70B and Google Translate as currently the best solution for EN-MK translation. We did not include any models from the same size in the testing as their results were suboptimal.

In order to leverage the instruction training your prompt should follow the alpaca format:

prompt = "<s> Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.\n\n### Instruction:\nTranslate the following text from {direction}.\n\n### Input:\n{sample}\n\n### Response:\n"
where {direction} is one of "English to Macedonian" or "Macedonian to English" and {sample} is the text you want translated.

Here's a full example of translating "Hi there, how's it going?"

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

tokenizer = AutoTokenizer.from_pretrained("trajkovnikola/MKLLM-7B-Translate")
model = AutoModelForCausalLM.from_pretrained("trajkovnikola/MKLLM-7B-Translate", torch_dtype=torch.bfloat16, device_map="auto")

sample = "Hi there, how's it going?"
query = f"<s> Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.\n\n### Instruction:\nTranslate the following text from English to Macedonian.\n\n### Input:\n{sample}\n\n### Response:\n"

model_input = tokenizer(query, return_tensors="pt").to("cuda")
with torch.no_grad():
  generated_ids = model.generate(**model_input,
                                 max_new_tokens=100,
                                 do_sample=False,
                                 repetition_penalty=1.1,
                                 )
print(tokenizer.decode(generated_ids[0][model_input["input_ids"].shape[1]:], skip_special_tokens=True))