Object namer (PE-Core B16, Core ML)

The image encoder of Meta's Perception Encoder PE-Core B16-224, converted to Core ML (fp16, iOS 18+) for the Spotword iOS app, which downloads it on request to name photographed objects on the device.

  • ObjectNamer.mlpackage: input image (RGB 224ร—224), output embedding (1024 floats, not unit length: scale it to unit length yourself; the L2 norm overflows in fp16 inside the model).
  • object-names.json / object-names.bin: text embeddings (float16, one row per name, then per wider word) of 3,496 everyday English nouns from WordNet, made with the model's text encoder and the prompts "a photo of a {}.", "a close-up photo of a {}.", "a photo of the {}.", "a {} on a table.", plus each name's word frequency (wordfreq Zipf) and wider words.

Changes from the original: weights converted from PyTorch to Core ML fp16; image normalisation built in; final L2 normalisation removed. Original model: https://github.com/facebookresearch/perception_models (Apache License 2.0, Copyright Meta Platforms, Inc. and affiliates).

Word frequencies: wordfreq by Robyn Speer, CC BY-SA 4.0 (https://github.com/rspeer/wordfreq). Noun list: Princeton WordNet 3.0 (WordNet license).

Downloads last month
1
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for ripehand/spotword-object-namer

Quantized
(2)
this model