Instructions to use ripehand/spotword-object-namer with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PerceptionEncoder
How to use ripehand/spotword-object-namer with PerceptionEncoder:
# Use any PE model as a vision encoder import core.vision_encoder.pe as pe model = pe.VisionTransformer.from_config("ripehand/spotword-object-namer", pretrained=True) - Notebooks
- Google Colab
- Kaggle
Object namer (PE-Core B16, Core ML)
The image encoder of Meta's Perception Encoder PE-Core B16-224, converted to Core ML (fp16, iOS 18+) for the Spotword iOS app, which downloads it on request to name photographed objects on the device.
ObjectNamer.mlpackage: inputimage(RGB 224ร224), outputembedding(1024 floats, not unit length: scale it to unit length yourself; the L2 norm overflows in fp16 inside the model).object-names.json/object-names.bin: text embeddings (float16, one row per name, then per wider word) of 3,496 everyday English nouns from WordNet, made with the model's text encoder and the prompts "a photo of a {}.", "a close-up photo of a {}.", "a photo of the {}.", "a {} on a table.", plus each name's word frequency (wordfreq Zipf) and wider words.
Changes from the original: weights converted from PyTorch to Core ML fp16; image normalisation built in; final L2 normalisation removed. Original model: https://github.com/facebookresearch/perception_models (Apache License 2.0, Copyright Meta Platforms, Inc. and affiliates).
Word frequencies: wordfreq by Robyn Speer, CC BY-SA 4.0 (https://github.com/rspeer/wordfreq). Noun list: Princeton WordNet 3.0 (WordNet license).
- Downloads last month
- 1
Model tree for ripehand/spotword-object-namer
Base model
facebook/PE-Core-B16-224