Instructions to use Ronin48LLC/abby-lora-adapter with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Ronin48LLC/abby-lora-adapter with PEFT:

from peft import PeftModel
from transformers import AutoModelForCausalLM

base_model = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-3.3-70B-Instruct")
model = PeftModel.from_pretrained(base_model, "Ronin48LLC/abby-lora-adapter")

Transformers

How to use Ronin48LLC/abby-lora-adapter with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Ronin48LLC/abby-lora-adapter")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("Ronin48LLC/abby-lora-adapter", dtype="auto")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use Ronin48LLC/abby-lora-adapter with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Ronin48LLC/abby-lora-adapter"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Ronin48LLC/abby-lora-adapter",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Ronin48LLC/abby-lora-adapter

SGLang

How to use Ronin48LLC/abby-lora-adapter with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Ronin48LLC/abby-lora-adapter" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Ronin48LLC/abby-lora-adapter",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Ronin48LLC/abby-lora-adapter" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Ronin48LLC/abby-lora-adapter",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Ronin48LLC/abby-lora-adapter with Docker Model Runner:
```
docker model run hf.co/Ronin48LLC/abby-lora-adapter
```

ABBY — Artifact, Ballistic, and Binary Yield

"I use my powers for good, not evil." — Abby Sciuto, NCIS

An open-source LoRA adapter fine-tuned for law enforcement forensic investigators.

ABBY is a QLoRA adapter trained on top of Meta Llama 3.3 70B Instruct, specialized for forensic analysis, evidence interpretation, ballistic assessment, digital artifact examination, and investigative reasoning.

Model Details

Field	Value
Base Model	meta-llama/Llama-3.3-70B-Instruct
Adapter Type	LoRA (QLoRA, 4-bit NF4)
LoRA Rank	64
Task	Forensic investigation, evidence analysis, investigative reasoning
Training Method	QLoRA fine-tuning via SFTTrainer (trl)
License	Apache 2.0
Maintainer	Ronin48LLC

Intended Use

ABBY is designed to assist certified forensic examiners, law enforcement investigators, and legal professionals with:

Digital forensics — artifact recovery, file system analysis, metadata examination
Ballistic analysis — wound pattern assessment, trajectory reconstruction
Binary/malware analysis — code review, threat identification
Chain of custody guidance — evidence handling best practices
Investigative reasoning — connecting evidence to conclusions

This model is intended for professional use in authorized law enforcement and forensic contexts only.

Usage

This is a PEFT LoRA adapter. Load it on top of the base model:

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_model = AutoModelForCausalLM.from_pretrained(
    "meta-llama/Llama-3.3-70B-Instruct",
    load_in_4bit=True,
    device_map="auto",
)
model = PeftModel.from_pretrained(base_model, "Ronin48LLC/abby-lora-adapter")
tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-3.3-70B-Instruct")

Note: Access to the base model requires accepting Meta's license on HuggingFace.

Training

Framework: PyTorch 2.4 + Transformers + TRL + PEFT
Quantization: 4-bit NF4 (bitsandbytes)
Hardware: NVIDIA A100 PCIe 80GB
Training steps: 78
Sequence length: 4096

Supporters

ABBY is community-funded. Every contribution keeps this project free and in the hands of investigators who need it.

Donor	Amount	Note
Ronin 48, LLC	N/A	Founding donor
Anonymous	$50	Thank you, Joe Sixpack

Want to support ABBY? Reach out to the maintainers.

Related Projects

Project	Description
SELMA	Criminal law and statute analysis
ATTICUS	Legal defense reasoning
BONES	Forensic pathology and osteology
BRUNO	Field operations and tactical support

License

Fine-tuned adapter weights are licensed under Apache 2.0. Base model weights are subject to the Meta Llama 3.1 Community License.

Downloads last month: 27

Model tree for Ronin48LLC/abby-lora-adapter

Base model

meta-llama/Llama-3.1-70B

Finetuned

meta-llama/Llama-3.3-70B-Instruct

Adapter

(181)

this model