pathcowork-grounder-pro-1T
version 1.3
This repository holds the LoRA weights of the PathCowork grounder trained on Kimi-K2.6 (PEFT adapter, r = 32, all linear layers). Load it on top of moonshotai/Kimi-K2.6.
Examples below: one H&E field and one question per request. The answer is a JSON list of polygons/polylines with integer coordinates on a 0–1000 grid over the image (x rightward, y downward), or There are none.
- A request can ask for one concept or several concepts at once.
- The model was trained on target concepts from five common organs.
- It can also ground concepts outside that list.
1. One view, three questions: single class, several classes, absent class
The same 1.0 mm skin field (1 µm/px) asked three different ways.
1a. Single class
Prompt:
H&E-stained skin section, 1 µm/px, field 1.0 mm wide. Find every one of the junctional melanocytic nests in this image and outline each.
Answer:
```json
[
{"polygon_2d": [[415, 325], [470, 315], [520, 340], [540, 390], [530, 445], [480, 455], [430, 430], [410, 380]], "label": "junctional melanocytic nests"}
]
```
1b. Several classes in one request
Prompt:
H&E-stained skin section, 1 µm/px, field 1.0 mm wide. Locate each of the following where it is present in this image: junctional melanocytic nests, lentiginous melanocytic proliferation, melanophages, dermal melanocytic nests.
Answer:
```json
[
{"polygon_2d": [[410, 320], [470, 310], [530, 340], [540, 400], [510, 450], [450, 450], [410, 400]], "label": "dermal melanocytic nests"},
{"polygon_2d": [[400, 0], [440, 0], [450, 60], [430, 110], [400, 100], [390, 50]], "label": "junctional melanocytic nests"},
{"polygon_2d": [[560, 380], [600, 380], [610, 440], [580, 460], [555, 420]], "label": "lentiginous melanocytic proliferation"}
]
```
1c. Absent class
Prompt:
H&E-stained skin section, 1 µm/px, field 1.0 mm wide. Outline every area of pagetoid spread of melanocytes in this image.
Answer:
There are none.
2. Five organs
Brain: pseudopalisading necrosis (2 µm/px)
Prompt:
H&E-stained brain section, 2 µm/px, field 2.0 mm wide. Outline every area of pseudopalisading necrosis in this image.
Answer:
```json
[
{"polygon_2d": [[0, 640], [1000, 0], [1000, 1000], [0, 1000]], "label": "pseudopalisading necrosis"}
]
```
Breast: ductal carcinoma in situ (1 µm/px)
Prompt:
H&E-stained breast section, 1 µm/px, field 1.0 mm wide. Outline every area of ductal carcinoma in situ in this image.
Answer:
```json
[
{"polygon_2d": [[470, 0], [575, 330], [575, 560], [480, 791], [330, 806], [0, 721], [0, 5], [5, 0]], "label": "ductal carcinoma in situ"}
]
```
Lymph node: metastatic tumour focus (1 µm/px)
Prompt:
H&E-stained lymph node section, 1 µm/px, field 1.0 mm wide. Outline all metastatic tumour foci in this image.
Answer:
```json
[
{"polygon_2d": [[560, 415], [575, 500], [540, 590], [420, 625], [300, 600], [300, 545], [400, 450], [480, 405]], "label": "metastatic tumour foci"}
]
```
Prostate needle biopsy: Gleason pattern 3 (2 µm/px)
Prompt:
H&E-stained prostate needle biopsy section, 2 µm/px, field 2.0 mm wide. Find all Gleason pattern 3 carcinoma in this image and outline each area.
Answer:
```json
[
{"polygon_2d": [[999, 150], [999, 527], [950, 556], [279, 655], [24, 607], [0, 573], [0, 391], [974, 142]], "label": "Gleason pattern 3 carcinoma"}
]
```
Skin: dermal melanocytic nests (1 µm/px)
Prompt:
H&E-stained skin section, 1 µm/px, field 1.0 mm wide. Outline all dermal melanocytic nests in this image.
Answer:
```json
[
{"polygon_2d": [[420, 300], [480, 285], [545, 300], [565, 360], [555, 430], [500, 450], [440, 430], [415, 370]], "label": "dermal melanocytic nests"}
]
```
3. Out-of-distribution organs and concepts
Organs and concepts that were not in the training set, asked on whole-section views.
Bladder: papillary fronds (23 µm/px, whole section)
Prompt:
H&E-stained bladder section, 23 µm/px, field 23.6 mm wide. Find all papillary fronds in this image and outline each area.
Answer:
```json
[
{"polygon_2d": [[930, 0], [958, 245], [895, 400], [918, 590], [800, 615], [640, 665], [615, 700], [690, 840], [640, 830], [600, 700], [600, 420], [520, 150], [540, 90], [830, 0]], "label": "papillary fronds"}
]
```
Colon: normal colonic mucosa (21 µm/px, whole section)
Prompt:
H&E-stained colon section, 21 µm/px, field 21.5 mm wide. Find all normal colonic mucosa in this image and outline each area.
Answer:
```json
[
{"polygon_2d": [[999, 414], [999, 537], [959, 537], [895, 414], [831, 231], [905, 255], [959, 319]], "label": "normal colonic mucosa"},
{"polygon_2d": [[352, 531], [352, 555], [331, 600], [243, 631], [146, 643], [0, 650], [0, 573], [188, 463], [265, 463]], "label": "normal colonic mucosa"}
]
```
Lung: alveolar lung parenchyma (5 µm/px)
Prompt:
H&E-stained lung section, 5 µm/px, field 5.1 mm wide. Find all alveolar lung parenchyma in this image and outline each area.
Answer:
```json
[
{"polygon_2d": [[0, 0], [1000, 0], [1000, 1000], [440, 1000], [0, 290]], "label": "alveolar lung parenchyma"}
]
```
Stomach: normal gastric mucosa (19 µm/px, whole section)
Prompt:
H&E-stained stomach section, 19 µm/px, field 19.5 mm wide. Find all normal gastric mucosa in this image and outline each area.
Answer:
```json
[
{"polygon_2d": [[999, 200], [999, 265], [960, 300], [880, 320], [800, 330], [770, 350], [735, 355], [725, 320], [735, 280], [760, 250], [830, 230], [900, 215], [960, 195]], "label": "normal gastric mucosa"}
]
```
4. Trained organs and target concepts
- Brain (glioma): microvascular proliferation; pseudopalisading necrosis
- Breast: invasive carcinoma; ductal carcinoma in situ; normal breast epithelium; lymphocytic infiltrate; invasive tumour nuclei
- Lymph node: metastatic deposits / nodal metastases larger than 2 mm; metastatic tumour foci; lymph nodes; lymph node capsule (polyline)
- Prostate needle biopsy: prostatic adenocarcinoma; Gleason pattern 3 carcinoma; Gleason pattern 4 carcinoma; Gleason pattern 5 carcinoma
- Skin (melanocytic lesions): junctional melanocytic nests; dermal melanocytic nests; lentiginous melanocytic proliferation; confluent junctional melanocytic growth; pagetoid spread of melanocytes; dermal melanocytes showing maturation; dermal melanocytes without maturation; melanophages; papillary dermal fibroplasia; dermoepidermal junction (polyline)
Models are post-trained from the Kimi-K2.6 family.
License
Released under the Penn Academic Software License Agreement for PathCowork (internal research / non-commercial use; no redistribution without written permission from Penn). See LICENSE.
Model tree for zhihuanglab/pathcowork-grounder-pro-1T
Base model
moonshotai/Kimi-K2.6










