pathcowork-grounder-pro-1T

version 1.3

This repository holds the LoRA weights of the PathCowork grounder trained on Kimi-K2.6 (PEFT adapter, r = 32, all linear layers). Load it on top of moonshotai/Kimi-K2.6.

Examples below: one H&E field and one question per request. The answer is a JSON list of polygons/polylines with integer coordinates on a 0–1000 grid over the image (x rightward, y downward), or There are none.

  1. A request can ask for one concept or several concepts at once.
  2. The model was trained on target concepts from five common organs.
  3. It can also ground concepts outside that list.

1. One view, three questions: single class, several classes, absent class

The same 1.0 mm skin field (1 µm/px) asked three different ways.

1a. Single class

1a. Single class

Prompt:

H&E-stained skin section, 1 µm/px, field 1.0 mm wide. Find every one of the junctional melanocytic nests in this image and outline each.

Answer:

```json
[
    {"polygon_2d": [[415, 325], [470, 315], [520, 340], [540, 390], [530, 445], [480, 455], [430, 430], [410, 380]], "label": "junctional melanocytic nests"}
]
```

1b. Several classes in one request

1b. Several classes in one request

Prompt:

H&E-stained skin section, 1 µm/px, field 1.0 mm wide. Locate each of the following where it is present in this image: junctional melanocytic nests, lentiginous melanocytic proliferation, melanophages, dermal melanocytic nests.

Answer:

```json
[
    {"polygon_2d": [[410, 320], [470, 310], [530, 340], [540, 400], [510, 450], [450, 450], [410, 400]], "label": "dermal melanocytic nests"},
    {"polygon_2d": [[400, 0], [440, 0], [450, 60], [430, 110], [400, 100], [390, 50]], "label": "junctional melanocytic nests"},
    {"polygon_2d": [[560, 380], [600, 380], [610, 440], [580, 460], [555, 420]], "label": "lentiginous melanocytic proliferation"}
]
```

1c. Absent class

1c. Absent class

Prompt:

H&E-stained skin section, 1 µm/px, field 1.0 mm wide. Outline every area of pagetoid spread of melanocytes in this image.

Answer:

There are none.

2. Five organs

Brain: pseudopalisading necrosis (2 µm/px)

Brain: pseudopalisading necrosis (2 µm/px)

Prompt:

H&E-stained brain section, 2 µm/px, field 2.0 mm wide. Outline every area of pseudopalisading necrosis in this image.

Answer:

```json
[
    {"polygon_2d": [[0, 640], [1000, 0], [1000, 1000], [0, 1000]], "label": "pseudopalisading necrosis"}
]
```

Breast: ductal carcinoma in situ (1 µm/px)

Breast: ductal carcinoma in situ (1 µm/px)

Prompt:

H&E-stained breast section, 1 µm/px, field 1.0 mm wide. Outline every area of ductal carcinoma in situ in this image.

Answer:

```json
[
    {"polygon_2d": [[470, 0], [575, 330], [575, 560], [480, 791], [330, 806], [0, 721], [0, 5], [5, 0]], "label": "ductal carcinoma in situ"}
]
```

Lymph node: metastatic tumour focus (1 µm/px)

Lymph node: metastatic tumour focus (1 µm/px)

Prompt:

H&E-stained lymph node section, 1 µm/px, field 1.0 mm wide. Outline all metastatic tumour foci in this image.

Answer:

```json
[
    {"polygon_2d": [[560, 415], [575, 500], [540, 590], [420, 625], [300, 600], [300, 545], [400, 450], [480, 405]], "label": "metastatic tumour foci"}
]
```

Prostate needle biopsy: Gleason pattern 3 (2 µm/px)

Prostate needle biopsy: Gleason pattern 3 (2 µm/px)

Prompt:

H&E-stained prostate needle biopsy section, 2 µm/px, field 2.0 mm wide. Find all Gleason pattern 3 carcinoma in this image and outline each area.

Answer:

```json
[
    {"polygon_2d": [[999, 150], [999, 527], [950, 556], [279, 655], [24, 607], [0, 573], [0, 391], [974, 142]], "label": "Gleason pattern 3 carcinoma"}
]
```

Skin: dermal melanocytic nests (1 µm/px)

Skin: dermal melanocytic nests (1 µm/px)

Prompt:

H&E-stained skin section, 1 µm/px, field 1.0 mm wide. Outline all dermal melanocytic nests in this image.

Answer:

```json
[
    {"polygon_2d": [[420, 300], [480, 285], [545, 300], [565, 360], [555, 430], [500, 450], [440, 430], [415, 370]], "label": "dermal melanocytic nests"}
]
```

3. Out-of-distribution organs and concepts

Organs and concepts that were not in the training set, asked on whole-section views.

Bladder: papillary fronds (23 µm/px, whole section)

Bladder: papillary fronds (23 µm/px, whole section)

Prompt:

H&E-stained bladder section, 23 µm/px, field 23.6 mm wide. Find all papillary fronds in this image and outline each area.

Answer:

```json
[
    {"polygon_2d": [[930, 0], [958, 245], [895, 400], [918, 590], [800, 615], [640, 665], [615, 700], [690, 840], [640, 830], [600, 700], [600, 420], [520, 150], [540, 90], [830, 0]], "label": "papillary fronds"}
]
```

Colon: normal colonic mucosa (21 µm/px, whole section)

Colon: normal colonic mucosa (21 µm/px, whole section)

Prompt:

H&E-stained colon section, 21 µm/px, field 21.5 mm wide. Find all normal colonic mucosa in this image and outline each area.

Answer:

```json
[
    {"polygon_2d": [[999, 414], [999, 537], [959, 537], [895, 414], [831, 231], [905, 255], [959, 319]], "label": "normal colonic mucosa"},
    {"polygon_2d": [[352, 531], [352, 555], [331, 600], [243, 631], [146, 643], [0, 650], [0, 573], [188, 463], [265, 463]], "label": "normal colonic mucosa"}
]
```

Lung: alveolar lung parenchyma (5 µm/px)

Lung: alveolar lung parenchyma (5 µm/px)

Prompt:

H&E-stained lung section, 5 µm/px, field 5.1 mm wide. Find all alveolar lung parenchyma in this image and outline each area.

Answer:

```json
[
    {"polygon_2d": [[0, 0], [1000, 0], [1000, 1000], [440, 1000], [0, 290]], "label": "alveolar lung parenchyma"}
]
```

Stomach: normal gastric mucosa (19 µm/px, whole section)

Stomach: normal gastric mucosa (19 µm/px, whole section)

Prompt:

H&E-stained stomach section, 19 µm/px, field 19.5 mm wide. Find all normal gastric mucosa in this image and outline each area.

Answer:

```json
[
    {"polygon_2d": [[999, 200], [999, 265], [960, 300], [880, 320], [800, 330], [770, 350], [735, 355], [725, 320], [735, 280], [760, 250], [830, 230], [900, 215], [960, 195]], "label": "normal gastric mucosa"}
]
```

4. Trained organs and target concepts

  • Brain (glioma): microvascular proliferation; pseudopalisading necrosis
  • Breast: invasive carcinoma; ductal carcinoma in situ; normal breast epithelium; lymphocytic infiltrate; invasive tumour nuclei
  • Lymph node: metastatic deposits / nodal metastases larger than 2 mm; metastatic tumour foci; lymph nodes; lymph node capsule (polyline)
  • Prostate needle biopsy: prostatic adenocarcinoma; Gleason pattern 3 carcinoma; Gleason pattern 4 carcinoma; Gleason pattern 5 carcinoma
  • Skin (melanocytic lesions): junctional melanocytic nests; dermal melanocytic nests; lentiginous melanocytic proliferation; confluent junctional melanocytic growth; pagetoid spread of melanocytes; dermal melanocytes showing maturation; dermal melanocytes without maturation; melanophages; papillary dermal fibroplasia; dermoepidermal junction (polyline)

Models are post-trained from the Kimi-K2.6 family.

License

Released under the Penn Academic Software License Agreement for PathCowork (internal research / non-commercial use; no redistribution without written permission from Penn). See LICENSE.

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for zhihuanglab/pathcowork-grounder-pro-1T

Adapter
(14)
this model