---
library_name: onnxruntime
pipeline_tag: image-classification
tags:
- onnx
- image-classification
- medieval-manuscripts
- illumination-detection
- mobilenet
- mobilevit
- glam
- iiif
- cultural-heritage
- digital-humanities
- medieval-folio
- medieval
- medieval-illuminations
- MobileNet
- MobileVit
license: apache-2.0
datasets:
- ENC-PSL/BSICLE-medieval-illumination-folio-bin-class-dataset
base_model:
- timm/mobilenetv3_small_100.lamb_in1k
- timm/mobilenetv3_large_100.ra_in1k
- timm/mobilenetv2_100.ra_in1k
- apple/mobilevitv2-1.0-imagenet1k-256
---
# BSICLE — Binary System for Illuminated Folio Classification with Lightweight Engines
[](https://huggingface.co/spaces/ENC-PSL/Medieval-Illumination-Detector) [](https://odil-ai.github.io/medieval-illumination-detector/)
BSICLE (pronounced bé-si-cle ; /be.zikl/) is a family of lightweight binary models for classify **illuminated folios in medieval manuscripts**. Theses models are developed at the [École nationale des chartes – PSL](https://www.chartes.psl.eu/) in context of [O.D.I.L. project](https://projet.biblissima.fr/fr/appels-projets/projets-retenus/odil-objet-detection-illuminations).
These models classify manuscript pages as:
- **illuminated** (miniatures, historiated initials, decorated pages etc.)
- **non-illuminated** (plain text folio, printer marks, tables, cover, blank folios etc.)
# Use cases
Models are optimized to run **locally (CPU) or in the browser** using **edge-compatibility architecture** (MobileNet, MobileViT) and **ONNX inference** for exemple to build **IIIF filter pipelines** or to build **specialized corpora**.
Try the demo web application on [HF Spaces](https://huggingface.co/spaces/ENC-PSL/Medieval-Illumination-Detector) or on [GH Pages](https://odil-ai.github.io/medieval-illumination-detector/)
# Models & Results
The finetuned models available in this repository are based on following architecture:
- [MobileNetV2](https://huggingface.co/timm/mobilenetv2_100.ra_in1k)
- [MobileNetV3](https://huggingface.co/timm/mobilenetv3_small_100.lamb_in1k) (small and large version)
- [MobileViT v2](https://huggingface.co/apple/mobilevitv2-1.0-imagenet1k-256)
| Architecture | Validation Accuracy | Test Accuracy |
|--------------|--------------------:|--------------:|
| MobileNetV2 | 0.995 | 0.982 |
| MobileNetV3 Small | 0.991 | 0.968 |
| MobileNetV3 Large | 1.0 | 0.986 |
| MobileViT v2 | 0.995 | 0.977 |
> These results should be interpreted with care. Although the models reach very high scores on the current splits, the task may be partially dataset-dependent.
# Labels
| Label ID | Label |
|---------|------|
| 0 | non_illuminated |
| 1 | illuminated |
# Dataset
Training data comes from: [ENC-PSL/BSICLE-medieval-illumination-folio-bin-class-dataset](https://huggingface.co/datasets/ENC-PSL/BSICLE-medieval-illumination-folio-bin-class-dataset)
## Distribution of data
- illuminated
- train: 519
- dev : 112
- test : 111
- non_illuminated
- train: 519
- dev : 111
- test : 112
> **Data augmentation.** During training, data augmentation was applied to the training split only in order to improve robustness and reduce overfitting.
> The augmentation pipeline included random horizontal flips, small random rotations up to 5°, and light color jittering with brightness `0.12`, contrast `0.12`, saturation `0.08`, and hue `0.02`.
> Validation and test images were evaluated without augmentation.
# What counts as "illuminated"?
### Positive (illuminated)
Examples include:
- miniatures
- historiated initials
- decorative initials
- scientific diagrams
- maps
- decorated manuscript pages
### Negative (non-illuminated)
Examples include:
- plain text folios
- marginal decorations without images
- printer marks
- tables
- cover
- blank folios
- rubricated text without illumination
### Examples
| Illuminated | Not Illuminated |
|:-----------:|:---------------:|
| | |
| | |
| | |
| | |
| | |
| | |
# Usage
## Python — ONNX local
```bash
pip install onnxruntime pillow numpy
```
```python
import json
import numpy as np
import onnxruntime as ort
from PIL import Image
from pathlib import Path
# Model list to test.
# Check if the models are stored
# in your expected directory structure
# (e.g. './Artefacts/{model_name}/onnx/model.onnx')
MODELS = [
"mobilenetv2",
"mobilenetv3_large",
"mobilenetv3_small",
"mobilevitv2",
]
ARTEFACTS_DIR = Path("./Artefacts")
# Test images
IMAGE_PATHS = {
"illumination": Path("./dataset/test/illustration/gahom_0020__fdf0ee350c94.jpg"),
"non_illumination": Path("./dataset/test/non_illustration/CREMMA-Medieval-LAT_00007.jpg"),
}
def softmax(logits: np.ndarray) -> np.ndarray:
"""Softmax function to convert logits to probabilities.
:param logits: logits array
:type logits: np.ndarray
:return: probabilities array
:rtype: np.ndarray
"""
logits = logits.astype(np.float32)
exp = np.exp(logits - logits.max())
return exp / exp.sum()
def load_json(path: Path) -> dict:
"""Load a JSON file and return its content as a dictionary.
:param path: path to the JSON file
:type path: Path
:return: content of the JSON file as a dictionary
:rtype: dict
"""
if not path.exists():
raise FileNotFoundError(f"File not founded: {path}")
return json.loads(path.read_text())
def preprocess_image(image_path: Path, pre: dict) -> np.ndarray:
"""Preprocess the image according to the provided configuration.
:param image_path: path to the image file
:type image_path: Path
:param pre: preprocessing configuration (expects keys 'img_size', 'mean', 'std
:type pre: dict
:return: preprocessed image as a numpy array ready for model input
:rtype: np.ndarray
"""
if not image_path.exists():
raise FileNotFoundError(f"Image not founded: {image_path}")
img_size = pre["img_size"]
mean = np.array(pre["mean"], dtype=np.float32)
std = np.array(pre["std"], dtype=np.float32)
img = Image.open(image_path).convert("RGB").resize((img_size, img_size))
x = np.asarray(img).astype(np.float32) / 255.0
x = (x - mean) / std
x = x.transpose(2, 0, 1)[None].astype(np.float32)
return x
def predict(model_name: str, image_path: Path) -> dict:
"""Run inference on a single model and return the results.
:param model_name: name of the model to test
:type model_name: str
:param image_path: path to the image file to test
:type image_path: Path
:return: dictionary containing the prediction results and probabilities
:rtype: dict
"""
run = ARTEFACTS_DIR / model_name
cfg = load_json(run / "inference_config.json")
pre = load_json(run / "preprocess.json")
model_path = run / "onnx" / "model.onnx"
if not model_path.exists():
raise FileNotFoundError(f"ONNX model not founded: {model_path}")
x = preprocess_image(image_path, pre)
sess = ort.InferenceSession(str(model_path))
input_name = cfg.get("input_name")
if input_name is None:
input_name = sess.get_inputs()[0].name
output = sess.run(None, {input_name: x})[0]
# Cas standard : shape (1, 2)
logits = output[0]
probs = softmax(logits)
p_illu = float(probs[1])
positive_label = cfg.get("positive_label", "illumination")
negative_label = cfg.get("negative_label", "non_illumination")
threshold = float(cfg.get("threshold", 0.5))
label = positive_label if p_illu >= threshold else negative_label
return {
"model": model_name,
"image": str(image_path),
"label": label,
"p_illustration": p_illu,
"probs": probs.tolist(),
"threshold": threshold,
}
def main():
"""Main function to run the tests on all models and images."""
for image_type, image_path in IMAGE_PATHS.items():
print("=" * 80)
print(f"Image expected: {image_type}")
print(f"Image: {image_path}")
print("=" * 80)
for model_name in MODELS:
try:
result = predict(model_name, image_path)
print(
f"{result['model']:<20} "
f"=> {result['label']:<18} "
f"p_illu={result['p_illustration']:.4f} "
f"probs={result['probs']}"
)
except Exception as e:
print(f"{model_name:<20} → ERROR: {e}")
print()
if __name__ == "__main__":
main()
```
## Python — ONNX from Hugging Face
```bash
pip install huggingface_hub onnxruntime pillow numpy
```
```python
import json
import numpy as np
import onnxruntime as ort
from PIL import Image
from pathlib import Path
from huggingface_hub import snapshot_download
# Repository HF that contains the ONNX models and their configs
REPO_ID = "ENC-PSL/BSICLE"
MODELS = [
"mobilenetv2",
"mobilenetv3_large",
"mobilenetv3_small",
"mobilevitv2",
]
IMAGE_PATHS = {
"illumination": Path("./dataset/test/illustration/gahom_0020__fdf0ee350c94.jpg"),
"non_illumination": Path("./dataset/test/non_illustration/CREMMA-Medieval-LAT_00007.jpg"),
}
def download_models(repo_id: str, model_names: list[str]) -> Path:
"""download models and their configs from HF Hub, and return the local path to the snapshot
:param repo_id: repository id on HF Hub
:type repo_id: str
:param model_names: list of model names to download (e.g. ["mobilenetv2", "mobilenetv3_large"])
:return: local path to the snapshot containing the models and their configs
:rtype: Path
"""
allow_patterns = []
for model_name in model_names:
allow_patterns.extend([
f"{model_name}/onnx/model.onnx",
f"{model_name}/preprocess.json",
f"{model_name}/inference_config.json",
])
snapshot_path = snapshot_download(
repo_id=repo_id,
allow_patterns=allow_patterns,
)
return Path(snapshot_path)
def load_json(path: Path) -> dict:
"""Load a JSON file and return its content as a dictionary.
:param path: path to the JSON file
:type path: Path
:return: content of the JSON file as a dictionary
:rtype: dict
"""
if not path.exists():
raise FileNotFoundError(f"Fichier introuvable : {path}")
return json.loads(path.read_text())
def softmax(logits: np.ndarray) -> np.ndarray:
"""Softmax function to convert logits to probabilities.
:param logits: logits array
:type logits: np.ndarray
:return: probabilities array
:rtype: np.ndarray
"""
logits = logits.astype(np.float32)
exp = np.exp(logits - logits.max())
return exp / exp.sum()
def preprocess_image(image_path: Path, pre: dict) -> np.ndarray:
"""Preprocess the image according to the provided configuration.
:param image_path: path to the image file
:type image_path: Path
:param pre: preprocessing configuration (expects keys 'img_size', 'mean', 'std
:type pre: dict
:return: preprocessed image as a numpy array ready for model input
:rtype: np.ndarray
"""
if not image_path.exists():
raise FileNotFoundError(f"Image not founded: {image_path}")
img_size = int(pre["img_size"])
mean = np.array(pre["mean"], dtype=np.float32)
std = np.array(pre["std"], dtype=np.float32)
img = Image.open(image_path).convert("RGB").resize((img_size, img_size))
x = np.asarray(img).astype(np.float32) / 255.0
x = (x - mean) / std
x = x.transpose(2, 0, 1)[None].astype(np.float32)
return x
def get_labels(cfg: dict) -> list[str]:
"""Get the list of class labels from the configuration dictionary.
:param cfg: configuration dictionary that may contain class labels in different keys
:type cfg: dict
:return: list of class labels
:rtype: list[str]
"""
if "class_names" in cfg:
return cfg["class_names"]
if "labels" in cfg:
return cfg["labels"]
if "id2label" in cfg:
id2label = cfg["id2label"]
return [
id2label[str(i)] if str(i) in id2label else id2label[i]
for i in range(len(id2label))
]
return [
cfg.get("negative_label", "non_illumination"),
cfg.get("positive_label", "illumination"),
]
def get_positive_index(labels: list[str], positive_label: str) -> int:
"""Get the index of the positive label in the labels list.
:param labels: list of class labels
:type labels: list[str]
:param positive_label: name of the positive label
:type positive_label: str
:return: index of the positive label
:rtype: int
"""
if positive_label in labels:
return labels.index(positive_label)
if len(labels) > 1:
return 1
raise ValueError(
f"Cannot determine positive index: positive_label={positive_label!r} not in labels={labels}"
)
def predict(model_dir: Path, image_path: Path) -> dict:
"""Run inference on a single model and return the results.
:param model_dir: path to the model directory
:type model_dir: Path
:param image_path: path to the image file to test
:type image_path: Path
:return: dictionary containing the prediction results and probabilities
:rtype: dict
"""
cfg = load_json(model_dir / "inference_config.json")
pre = load_json(model_dir / "preprocess.json")
model_path = model_dir / "onnx" / "model.onnx"
if not model_path.exists():
raise FileNotFoundError(f"ONNX model not founded: {model_path}")
x = preprocess_image(image_path, pre)
sess = ort.InferenceSession(str(model_path))
input_name = cfg.get("input_name")
if input_name is None:
input_name = sess.get_inputs()[0].name
output = sess.run(None, {input_name: x})[0]
logits = output[0]
probs = softmax(logits)
labels = get_labels(cfg)
positive_label = cfg.get("positive_label", "illumination")
negative_label = cfg.get("negative_label", "non_illumination")
threshold = float(cfg.get("threshold", 0.5))
positive_idx = get_positive_index(labels, positive_label)
argmax_idx = int(np.argmax(probs))
argmax_label = labels[argmax_idx]
argmax_score = float(probs[argmax_idx])
p_illumination = float(probs[positive_idx])
threshold_label = positive_label if p_illumination >= threshold else negative_label
probs_by_label = {
labels[i]: float(probs[i])
for i in range(len(labels))
}
return {
"label_threshold": threshold_label,
"p_illumination": p_illumination,
"threshold": threshold,
"positive_idx": positive_idx,
"argmax_idx": argmax_idx,
"argmax_label": argmax_label,
"score_argmax": argmax_score,
"labels": labels,
"probs": probs.tolist(),
"probs_by_label": probs_by_label,
}
def main():
"""Main function to run the tests on all models and images."""
snapshot_root = download_models(REPO_ID, MODELS)
print(f"Model downloaded in: {snapshot_root}")
print()
for image_type, image_path in IMAGE_PATHS.items():
print("=" * 100)
print(f"Image expected: {image_type}")
print(f"Image: {image_path}")
print("=" * 100)
for model_name in MODELS:
model_dir = snapshot_root / model_name
try:
result = predict(model_dir, image_path)
print(
f"{model_name:<20} "
f"=> predicted={result['label_threshold']:<18} "
f"p_illumination={result['p_illumination']:.4f} "
f"argmax={result['argmax_idx']}:{result['argmax_label']:<18} "
f"score={result['score_argmax']:.4f} "
f"probs={result['probs_by_label']}"
)
except Exception as e:
print(f"{model_name:<20} => ERROR : {e}")
print()
if __name__ == "__main__":
main()
```
## Python — PyTorch / non-ONNX local & Hugging Face
```bash
pip install torch torchvision pillow numpy timm
```
Use the same code as above, just change the function `load_model`.
```python
def load_model(run: Path) -> torch.nn.Module:
"""Load a PyTorch model from a checkpoint.
:param run: path to the model run directory
:type run: Path
:return: loaded PyTorch model
:rtype: torch.nn.Module
"""
checkpoint_path = run / "checkpoints" / "best.pt"
if not checkpoint_path.exists():
raise FileNotFoundError(f"Checkpoint introuvable : {checkpoint_path}")
model_name = run.name
if model_name in {"mobilenetv2", "mobilenet_v2"}:
model = models.mobilenet_v2(weights=None)
model.classifier[-1] = torch.nn.Linear(model.classifier[-1].in_features, 2)
elif model_name in {"mobilenetv3_large", "mobilenet_v3_large"}:
model = models.mobilenet_v3_large(weights=None)
model.classifier[-1] = torch.nn.Linear(model.classifier[-1].in_features, 2)
elif model_name in {"mobilenetv3_small", "mobilenet_v3_small"}:
model = models.mobilenet_v3_small(weights=None)
model.classifier[-1] = torch.nn.Linear(model.classifier[-1].in_features, 2)
elif model_name in {"mobilevitv2", "mobilevit_v2"}:
import timm
model = timm.create_model(
"mobilevitv2_050",
pretrained=False,
num_classes=2,
)
else:
raise ValueError(
f"Architecture non supportée : {model_name}. "
f"Architectures disponibles : mobilenetv2, mobilenetv3_large, "
f"mobilenetv3_small, mobilevitv2"
)
state = torch.load(checkpoint_path, map_location="cpu")
if isinstance(state, dict) and "state_dict" in state:
state = state["state_dict"]
if isinstance(state, dict) and "model_state_dict" in state:
state = state["model_state_dict"]
state = {
key.replace("module.", ""): value
for key, value in state.items()
}
model.load_state_dict(state)
model.eval()
return model
```
## JS (HF - ONNX)
```javascript
| 
|
| 
| 
|
| 
| 
|
| 
| 
|
| 
| 
|
| 
| 
|
# Usage
## Python — ONNX local
```bash
pip install onnxruntime pillow numpy
```
```python
import json
import numpy as np
import onnxruntime as ort
from PIL import Image
from pathlib import Path
# Model list to test.
# Check if the models are stored
# in your expected directory structure
# (e.g. './Artefacts/{model_name}/onnx/model.onnx')
MODELS = [
"mobilenetv2",
"mobilenetv3_large",
"mobilenetv3_small",
"mobilevitv2",
]
ARTEFACTS_DIR = Path("./Artefacts")
# Test images
IMAGE_PATHS = {
"illumination": Path("./dataset/test/illustration/gahom_0020__fdf0ee350c94.jpg"),
"non_illumination": Path("./dataset/test/non_illustration/CREMMA-Medieval-LAT_00007.jpg"),
}
def softmax(logits: np.ndarray) -> np.ndarray:
"""Softmax function to convert logits to probabilities.
:param logits: logits array
:type logits: np.ndarray
:return: probabilities array
:rtype: np.ndarray
"""
logits = logits.astype(np.float32)
exp = np.exp(logits - logits.max())
return exp / exp.sum()
def load_json(path: Path) -> dict:
"""Load a JSON file and return its content as a dictionary.
:param path: path to the JSON file
:type path: Path
:return: content of the JSON file as a dictionary
:rtype: dict
"""
if not path.exists():
raise FileNotFoundError(f"File not founded: {path}")
return json.loads(path.read_text())
def preprocess_image(image_path: Path, pre: dict) -> np.ndarray:
"""Preprocess the image according to the provided configuration.
:param image_path: path to the image file
:type image_path: Path
:param pre: preprocessing configuration (expects keys 'img_size', 'mean', 'std
:type pre: dict
:return: preprocessed image as a numpy array ready for model input
:rtype: np.ndarray
"""
if not image_path.exists():
raise FileNotFoundError(f"Image not founded: {image_path}")
img_size = pre["img_size"]
mean = np.array(pre["mean"], dtype=np.float32)
std = np.array(pre["std"], dtype=np.float32)
img = Image.open(image_path).convert("RGB").resize((img_size, img_size))
x = np.asarray(img).astype(np.float32) / 255.0
x = (x - mean) / std
x = x.transpose(2, 0, 1)[None].astype(np.float32)
return x
def predict(model_name: str, image_path: Path) -> dict:
"""Run inference on a single model and return the results.
:param model_name: name of the model to test
:type model_name: str
:param image_path: path to the image file to test
:type image_path: Path
:return: dictionary containing the prediction results and probabilities
:rtype: dict
"""
run = ARTEFACTS_DIR / model_name
cfg = load_json(run / "inference_config.json")
pre = load_json(run / "preprocess.json")
model_path = run / "onnx" / "model.onnx"
if not model_path.exists():
raise FileNotFoundError(f"ONNX model not founded: {model_path}")
x = preprocess_image(image_path, pre)
sess = ort.InferenceSession(str(model_path))
input_name = cfg.get("input_name")
if input_name is None:
input_name = sess.get_inputs()[0].name
output = sess.run(None, {input_name: x})[0]
# Cas standard : shape (1, 2)
logits = output[0]
probs = softmax(logits)
p_illu = float(probs[1])
positive_label = cfg.get("positive_label", "illumination")
negative_label = cfg.get("negative_label", "non_illumination")
threshold = float(cfg.get("threshold", 0.5))
label = positive_label if p_illu >= threshold else negative_label
return {
"model": model_name,
"image": str(image_path),
"label": label,
"p_illustration": p_illu,
"probs": probs.tolist(),
"threshold": threshold,
}
def main():
"""Main function to run the tests on all models and images."""
for image_type, image_path in IMAGE_PATHS.items():
print("=" * 80)
print(f"Image expected: {image_type}")
print(f"Image: {image_path}")
print("=" * 80)
for model_name in MODELS:
try:
result = predict(model_name, image_path)
print(
f"{result['model']:<20} "
f"=> {result['label']:<18} "
f"p_illu={result['p_illustration']:.4f} "
f"probs={result['probs']}"
)
except Exception as e:
print(f"{model_name:<20} → ERROR: {e}")
print()
if __name__ == "__main__":
main()
```
## Python — ONNX from Hugging Face
```bash
pip install huggingface_hub onnxruntime pillow numpy
```
```python
import json
import numpy as np
import onnxruntime as ort
from PIL import Image
from pathlib import Path
from huggingface_hub import snapshot_download
# Repository HF that contains the ONNX models and their configs
REPO_ID = "ENC-PSL/BSICLE"
MODELS = [
"mobilenetv2",
"mobilenetv3_large",
"mobilenetv3_small",
"mobilevitv2",
]
IMAGE_PATHS = {
"illumination": Path("./dataset/test/illustration/gahom_0020__fdf0ee350c94.jpg"),
"non_illumination": Path("./dataset/test/non_illustration/CREMMA-Medieval-LAT_00007.jpg"),
}
def download_models(repo_id: str, model_names: list[str]) -> Path:
"""download models and their configs from HF Hub, and return the local path to the snapshot
:param repo_id: repository id on HF Hub
:type repo_id: str
:param model_names: list of model names to download (e.g. ["mobilenetv2", "mobilenetv3_large"])
:return: local path to the snapshot containing the models and their configs
:rtype: Path
"""
allow_patterns = []
for model_name in model_names:
allow_patterns.extend([
f"{model_name}/onnx/model.onnx",
f"{model_name}/preprocess.json",
f"{model_name}/inference_config.json",
])
snapshot_path = snapshot_download(
repo_id=repo_id,
allow_patterns=allow_patterns,
)
return Path(snapshot_path)
def load_json(path: Path) -> dict:
"""Load a JSON file and return its content as a dictionary.
:param path: path to the JSON file
:type path: Path
:return: content of the JSON file as a dictionary
:rtype: dict
"""
if not path.exists():
raise FileNotFoundError(f"Fichier introuvable : {path}")
return json.loads(path.read_text())
def softmax(logits: np.ndarray) -> np.ndarray:
"""Softmax function to convert logits to probabilities.
:param logits: logits array
:type logits: np.ndarray
:return: probabilities array
:rtype: np.ndarray
"""
logits = logits.astype(np.float32)
exp = np.exp(logits - logits.max())
return exp / exp.sum()
def preprocess_image(image_path: Path, pre: dict) -> np.ndarray:
"""Preprocess the image according to the provided configuration.
:param image_path: path to the image file
:type image_path: Path
:param pre: preprocessing configuration (expects keys 'img_size', 'mean', 'std
:type pre: dict
:return: preprocessed image as a numpy array ready for model input
:rtype: np.ndarray
"""
if not image_path.exists():
raise FileNotFoundError(f"Image not founded: {image_path}")
img_size = int(pre["img_size"])
mean = np.array(pre["mean"], dtype=np.float32)
std = np.array(pre["std"], dtype=np.float32)
img = Image.open(image_path).convert("RGB").resize((img_size, img_size))
x = np.asarray(img).astype(np.float32) / 255.0
x = (x - mean) / std
x = x.transpose(2, 0, 1)[None].astype(np.float32)
return x
def get_labels(cfg: dict) -> list[str]:
"""Get the list of class labels from the configuration dictionary.
:param cfg: configuration dictionary that may contain class labels in different keys
:type cfg: dict
:return: list of class labels
:rtype: list[str]
"""
if "class_names" in cfg:
return cfg["class_names"]
if "labels" in cfg:
return cfg["labels"]
if "id2label" in cfg:
id2label = cfg["id2label"]
return [
id2label[str(i)] if str(i) in id2label else id2label[i]
for i in range(len(id2label))
]
return [
cfg.get("negative_label", "non_illumination"),
cfg.get("positive_label", "illumination"),
]
def get_positive_index(labels: list[str], positive_label: str) -> int:
"""Get the index of the positive label in the labels list.
:param labels: list of class labels
:type labels: list[str]
:param positive_label: name of the positive label
:type positive_label: str
:return: index of the positive label
:rtype: int
"""
if positive_label in labels:
return labels.index(positive_label)
if len(labels) > 1:
return 1
raise ValueError(
f"Cannot determine positive index: positive_label={positive_label!r} not in labels={labels}"
)
def predict(model_dir: Path, image_path: Path) -> dict:
"""Run inference on a single model and return the results.
:param model_dir: path to the model directory
:type model_dir: Path
:param image_path: path to the image file to test
:type image_path: Path
:return: dictionary containing the prediction results and probabilities
:rtype: dict
"""
cfg = load_json(model_dir / "inference_config.json")
pre = load_json(model_dir / "preprocess.json")
model_path = model_dir / "onnx" / "model.onnx"
if not model_path.exists():
raise FileNotFoundError(f"ONNX model not founded: {model_path}")
x = preprocess_image(image_path, pre)
sess = ort.InferenceSession(str(model_path))
input_name = cfg.get("input_name")
if input_name is None:
input_name = sess.get_inputs()[0].name
output = sess.run(None, {input_name: x})[0]
logits = output[0]
probs = softmax(logits)
labels = get_labels(cfg)
positive_label = cfg.get("positive_label", "illumination")
negative_label = cfg.get("negative_label", "non_illumination")
threshold = float(cfg.get("threshold", 0.5))
positive_idx = get_positive_index(labels, positive_label)
argmax_idx = int(np.argmax(probs))
argmax_label = labels[argmax_idx]
argmax_score = float(probs[argmax_idx])
p_illumination = float(probs[positive_idx])
threshold_label = positive_label if p_illumination >= threshold else negative_label
probs_by_label = {
labels[i]: float(probs[i])
for i in range(len(labels))
}
return {
"label_threshold": threshold_label,
"p_illumination": p_illumination,
"threshold": threshold,
"positive_idx": positive_idx,
"argmax_idx": argmax_idx,
"argmax_label": argmax_label,
"score_argmax": argmax_score,
"labels": labels,
"probs": probs.tolist(),
"probs_by_label": probs_by_label,
}
def main():
"""Main function to run the tests on all models and images."""
snapshot_root = download_models(REPO_ID, MODELS)
print(f"Model downloaded in: {snapshot_root}")
print()
for image_type, image_path in IMAGE_PATHS.items():
print("=" * 100)
print(f"Image expected: {image_type}")
print(f"Image: {image_path}")
print("=" * 100)
for model_name in MODELS:
model_dir = snapshot_root / model_name
try:
result = predict(model_dir, image_path)
print(
f"{model_name:<20} "
f"=> predicted={result['label_threshold']:<18} "
f"p_illumination={result['p_illumination']:.4f} "
f"argmax={result['argmax_idx']}:{result['argmax_label']:<18} "
f"score={result['score_argmax']:.4f} "
f"probs={result['probs_by_label']}"
)
except Exception as e:
print(f"{model_name:<20} => ERROR : {e}")
print()
if __name__ == "__main__":
main()
```
## Python — PyTorch / non-ONNX local & Hugging Face
```bash
pip install torch torchvision pillow numpy timm
```
Use the same code as above, just change the function `load_model`.
```python
def load_model(run: Path) -> torch.nn.Module:
"""Load a PyTorch model from a checkpoint.
:param run: path to the model run directory
:type run: Path
:return: loaded PyTorch model
:rtype: torch.nn.Module
"""
checkpoint_path = run / "checkpoints" / "best.pt"
if not checkpoint_path.exists():
raise FileNotFoundError(f"Checkpoint introuvable : {checkpoint_path}")
model_name = run.name
if model_name in {"mobilenetv2", "mobilenet_v2"}:
model = models.mobilenet_v2(weights=None)
model.classifier[-1] = torch.nn.Linear(model.classifier[-1].in_features, 2)
elif model_name in {"mobilenetv3_large", "mobilenet_v3_large"}:
model = models.mobilenet_v3_large(weights=None)
model.classifier[-1] = torch.nn.Linear(model.classifier[-1].in_features, 2)
elif model_name in {"mobilenetv3_small", "mobilenet_v3_small"}:
model = models.mobilenet_v3_small(weights=None)
model.classifier[-1] = torch.nn.Linear(model.classifier[-1].in_features, 2)
elif model_name in {"mobilevitv2", "mobilevit_v2"}:
import timm
model = timm.create_model(
"mobilevitv2_050",
pretrained=False,
num_classes=2,
)
else:
raise ValueError(
f"Architecture non supportée : {model_name}. "
f"Architectures disponibles : mobilenetv2, mobilenetv3_large, "
f"mobilenetv3_small, mobilevitv2"
)
state = torch.load(checkpoint_path, map_location="cpu")
if isinstance(state, dict) and "state_dict" in state:
state = state["state_dict"]
if isinstance(state, dict) and "model_state_dict" in state:
state = state["model_state_dict"]
state = {
key.replace("module.", ""): value
for key, value in state.items()
}
model.load_state_dict(state)
model.eval()
return model
```
## JS (HF - ONNX)
```javascript