# Third-party notices, schemer `schemer` is built for commercial use. It derives from the components below, each used under a license that permits commercial use and derivative works. Their license terms apply to those components; the attributions here satisfy them. Nothing in the Desert Ant Labs Source-Available License overrides them. ## Pretrained backbone ### mmBERT-base, Johns Hopkins University (CLSP) - [`jhu-clsp/mmBERT-base`](https://huggingface.co/jhu-clsp/mmBERT-base), **MIT**. Paper: [arXiv:2509.06888](https://arxiv.org/abs/2509.06888). - Use in `schemer`: fine-tuned. The distributed weights are modified from the upstream weights: trained for structured extraction and compressed for on-device use. They are not a drop-in replacement for mmBERT-base. - mmBERT's own pretraining corpora are listed on its model card and are one hop upstream of this model; they are not reproduced here. ```bibtex @misc{marone2025mmbertmodernmultilingualencoder, title = {mmBERT: A Modern Multilingual Encoder with Annealed Language Learning}, author = {Marc Marone and Orion Weller and William Fleshman and Eugene Yang and Dawn Lawrie and Benjamin Van Durme}, year = {2025}, eprint = {2509.06888}, archivePrefix = {arXiv}, primaryClass = {cs.CL}, url = {https://arxiv.org/abs/2509.06888}, } ``` ### ModernBERT, Answer.AI and LightOn (architecture lineage) - [`AnswerDotAI/ModernBERT`](https://github.com/AnswerDotAI/ModernBERT), **Apache-2.0**. Paper: [arXiv:2412.13663](https://arxiv.org/abs/2412.13663). - mmBERT is a ModernBERT-architecture model. No ModernBERT weights are used directly. ## Tokenizer - The shipped tokenizer (`schemer_tokenizer.bin`) is re-encoded from `tokenizer.json` in `jhu-clsp/mmBERT-base`, which is distributed under that repository's **MIT** license. - mmBERT's model card states that its tokenizer is the **Gemma 2** tokenizer ([`google/gemma-2-2b`](https://huggingface.co/google/gemma-2-2b)), released by Google under the [Gemma Terms of Use](https://ai.google.dev/gemma/terms). Those terms define "Gemma" as the language models, trained weights and parameters listed in their Appendix. Desert Ant Labs obtained the vocabulary through mmBERT-base's MIT release and did not use any Gemma model or weights; the origin is recorded here so a reader can trace the lineage. - [`huggingface/tokenizers`](https://github.com/huggingface/tokenizers), **Apache-2.0**: used at build time to read the upstream tokenizer and to verify the shipped one against it. Not linked into the SDK. ## Training data The datasets below are named because their license asks for attribution or because we consider naming them the honest thing to do. None of them is redistributed in this repository, and the model does not reproduce them. - [`wikimedia/wikipedia`](https://huggingface.co/datasets/wikimedia/wikipedia), **CC BY-SA 3.0** and **GFDL** as declared on the dataset card (Wikipedia text is contributed under CC BY-SA). Role: training data. Attribution: Wikipedia contributors. The remaining training data is proprietary to Desert Ant Labs. ## Build and export tooling Used to train, convert and package the model. None of these ship inside the artifacts in this repository. - [`pytorch/pytorch`](https://github.com/pytorch/pytorch), **BSD-3-Clause**: training and export. - [`huggingface/transformers`](https://github.com/huggingface/transformers), **Apache-2.0**: loading the backbone. - [`google-ai-edge/ai-edge-torch`](https://github.com/google-ai-edge/ai-edge-torch) and [`google-ai-edge/LiteRT`](https://github.com/google-ai-edge/LiteRT), **Apache-2.0**: the `.tflite` exports. - [`apple/coremltools`](https://github.com/apple/coremltools), **BSD-3-Clause**: the `.mlmodelc` exports. ## License notice retention For each Apache-2.0, MIT and BSD upstream above, the copyright holder and the license text as the license requires: | Component | Copyright | License text | |---|---|---| | mmBERT-base | Johns Hopkins University, Center for Language and Speech Processing (license declared as MIT on the model card) | [MIT](https://opensource.org/license/mit) | | ModernBERT | Answer.AI and LightOn | [Apache-2.0](https://github.com/AnswerDotAI/ModernBERT/blob/main/LICENSE) | | tokenizers | Hugging Face, Inc. | [Apache-2.0](https://github.com/huggingface/tokenizers/blob/main/LICENSE) | | transformers | 2018- The Hugging Face team | [Apache-2.0](https://github.com/huggingface/transformers/blob/main/LICENSE) | | PyTorch | 2016- Facebook, Inc. and contributors | [BSD-3-Clause](https://github.com/pytorch/pytorch/blob/main/LICENSE) | | ai-edge-torch | The AI Edge Torch Authors | [Apache-2.0](https://github.com/google-ai-edge/ai-edge-torch/blob/main/LICENSE) | | LiteRT | The LiteRT Authors | [Apache-2.0](https://github.com/google-ai-edge/LiteRT/blob/main/LICENSE) | | coremltools | 2020-2023 Apple Inc. | [BSD-3-Clause](https://github.com/apple/coremltools/blob/main/LICENSE.txt) | No Apache-2.0 upstream listed here ships a NOTICE file with additional attribution text.