darask0 commited on
Commit
eda6717
·
verified ·
1 Parent(s): 47ea602

ComfyUI custom node (unmerged fp32 LoRA + fp32 time path + data-space sampler), example workflows, GPU check

Browse files
.gitattributes CHANGED
@@ -40,3 +40,4 @@ images/training/000300.jpg filter=lfs diff=lfs merge=lfs -text
40
  images/training/000500.jpg filter=lfs diff=lfs merge=lfs -text
41
  images/training/000800.jpg filter=lfs diff=lfs merge=lfs -text
42
  images/training/init.jpg filter=lfs diff=lfs merge=lfs -text
 
 
40
  images/training/000500.jpg filter=lfs diff=lfs merge=lfs -text
41
  images/training/000800.jpg filter=lfs diff=lfs merge=lfs -text
42
  images/training/init.jpg filter=lfs diff=lfs merge=lfs -text
43
+ images/comfyui-check.jpg filter=lfs diff=lfs merge=lfs -text
README.md CHANGED
@@ -69,6 +69,26 @@ for r in range(4): # 1〜3 回で止め
69
  - time path の 3 モジュールは fp32 のまま使います。
70
  - CFG とネガティブプロンプトは使いません。プロンプトの形式(自然文・タグ列・JSON レイアウト、日本語の文字指定)は MOSS-Image と同じです。
71
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
72
  ## 結果
73
 
74
  左から teacher(50 step)/ lodestones 8 step 版を 4 step で使ったもの / このアダプタ(4 step)。同じ seed、評価用 17 プロンプト(`eval/eval_prompts.jsonl`、学習には未使用)。
 
69
  - time path の 3 モジュールは fp32 のまま使います。
70
  - CFG とネガティブプロンプトは使いません。プロンプトの形式(自然文・タグ列・JSON レイアウト、日本語の文字指定)は MOSS-Image と同じです。
71
 
72
+ ### ComfyUI
73
+
74
+ カスタムノード `comfyui/rapid-moss/` を同梱しています。ComfyUI 標準の `LoraLoaderModelOnly` は LoRA を重みにマージするので、このアダプタには使えません。このノードを使ってください。
75
+
76
+ 1. MOSS-Image 公式ノード([OpenMOSS/MOSS-Image](https://github.com/OpenMOSS/MOSS-Image) の `comfyui/moss-image/`)を入れます。
77
+ 2. このリポジトリの `comfyui/rapid-moss/` を `ComfyUI/custom_nodes/rapid-moss/` にコピーします。
78
+ 3. 上の 3 ファイルを `ComfyUI/models/rapid_moss/rapid-moss-4step/` に置きます。
79
+ 4. `comfyui/rapid-moss/example_workflows/rapid-moss-4step.json` を画面にドラッグ&ドロップし、HF Loader の `model_path` を MOSS-Image-SFT のフォルダにして実行します。
80
+
81
+ ノード **MOSS-Image Few-Step Adapter (rapid-moss)** は、次の処理をして MODEL と、専用の SAMPLER・SIGMAS を返します。
82
+
83
+ - LoRA をマージせず fp32 で足す
84
+ - time path を fp32 に置き換える
85
+
86
+ これを `BasicGuider`(CFG なし)と `SamplerCustomAdvanced` につなぎます。lodestones の 8 ステップ版も、同じ 3 ファイルを別フォルダに置けば同じノードで使えます。
87
+
88
+ 詳しい手順と注意点は [comfyui/rapid-moss/README.md](comfyui/rapid-moss/README.md) にあります。GPU での動作確認では、学習コードで作った画像とほぼ同じ絵になりました(下の図)。1024² 1 枚は 4.5〜6.9 秒(RTX PRO 6000、テキストエンコード・VAE 込み)です。
89
+
90
+ ![ComfyUI check](images/comfyui-check.jpg)
91
+
92
  ## 結果
93
 
94
  左から teacher(50 step)/ lodestones 8 step 版を 4 step で使ったもの / このアダプタ(4 step)。同じ seed、評価用 17 プロンプト(`eval/eval_prompts.jsonl`、学習には未使用)。
comfyui/rapid-moss/README.md ADDED
@@ -0,0 +1,85 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # rapid-moss for ComfyUI
2
+
3
+ MOSS-Image-SFT を **4 ステップ・CFG なし**で動かすカスタムノードです(rapid-moss の DMAD 4 ステップアダプタ。lodestones の 8 ステップ DMD も同じノードで使えます)。
4
+ ComfyUI 標準の `SamplerCustomAdvanced` と組み合わせて使います。
5
+
6
+ ## 必要なもの
7
+
8
+ | もの | 入手先 |
9
+ |---|---|
10
+ | ComfyUI | 新しめの版(`SamplerCustomAdvanced`、`BasicGuider`、`RandomNoise`、`EmptyFlux2LatentImage` があるもの。2026-10-16 の master で動作確認) |
11
+ | MOSS-Image 公式ノード | [OpenMOSS/MOSS-Image](https://github.com/OpenMOSS/MOSS-Image) の `comfyui/moss-image/` を `ComfyUI/custom_nodes/moss-image/` へコピー(ComfyUI Manager で「MOSS-Image」を検索してもよい) |
12
+ | MOSS-Image-SFT 本体 | [OpenMOSS-Team/MOSS-Image-SFT](https://huggingface.co/OpenMOSS-Team/MOSS-Image-SFT) のフォルダ一式(`transformer/`、`text_encoder/`、`tokenizer/`、`vae/`) |
13
+ | このノード | このフォルダ(`comfyui/rapid-moss/`)を `ComfyUI/custom_nodes/rapid-moss/` へコピー |
14
+ | アダプタ | [darask0/rapid-moss](https://huggingface.co/darask0/rapid-moss) の 3 ファイルを `ComfyUI/models/rapid_moss/rapid-moss-4step/` へ |
15
+
16
+ ```bash
17
+ cd ComfyUI
18
+ hf download OpenMOSS-Team/MOSS-Image-SFT --local-dir /path/to/MOSS-Image-SFT --exclude "assets/*"
19
+ hf download darask0/rapid-moss lora.safetensors time_embedding.safetensors dataspace.safetensors \
20
+ --local-dir models/rapid_moss/rapid-moss-4step
21
+ hf download darask0/rapid-moss --include "comfyui/rapid-moss/*" --local-dir /tmp/rapid-moss
22
+ cp -r /tmp/rapid-moss/comfyui/rapid-moss custom_nodes/rapid-moss
23
+ # 任意: lodestones の 8 ステップ版も使う場合
24
+ hf download lodestones/MOSS-Image-SFT-8-steps-dmd lora.safetensors time_embedding.safetensors dataspace.safetensors \
25
+ --local-dir models/rapid_moss/lodestones-8step
26
+ ```
27
+
28
+ `models/rapid_moss/` の下の、3 ファイル(`lora.safetensors`、`time_embedding.safetensors`、`dataspace.safetensors`)を含むフォルダが、ノードの `adapter` の選択肢に出ます。
29
+
30
+ ## 使い方
31
+
32
+ 1. ComfyUI を再起動し、`example_workflows/rapid-moss-4step.json` を画面にドラッグ&ドロップします。
33
+ 2. **MOSS-Image HF Loader (SFT)** の `model_path` に、MOSS-Image-SFT のフォルダを指定します。
34
+ 3. **Prompt** を書き換えて **実行** します。
35
+
36
+ ノードのつなぎ方:
37
+
38
+ ```
39
+ MOSS-Image HF Loader ─ MODEL ─▶ MOSS-Image Few-Step Adapter (rapid-moss) ─ model ──▶ BasicGuider ─▶ SamplerCustomAdvanced ─▶ VAE Decode ─▶ Save Image
40
+ │ ├ sampler ───────────────────────────▲ (sampler)
41
+ ├ CLIP ─▶ CLIPTextEncode(プロンプト)─▶ BasicGuider └ sigmas ────────────────────────────▲ (sigmas)
42
+ └ VAE ─────────────────────────────────────────────────────────────▶ VAE Decode
43
+ RandomNoise ─▶ SamplerCustomAdvanced (noise) EmptyFlux2LatentImage ─▶ SamplerCustomAdvanced (latent_image)
44
+ ```
45
+
46
+ API で投げる場合は `example_workflows/rapid-moss-4step-api.json` を使います(`model_path` を書き換えてください)。
47
+
48
+ ## ノードの設定
49
+
50
+ | 入力 | 説明 |
51
+ |---|---|
52
+ | `adapter` | `models/rapid_moss/` の下のフォルダ名 |
53
+ | `steps` | ネットワークの計算回数。rapid-moss は 1〜4(4 推奨)、lodestones は 1〜8 |
54
+ | `lora_strength` | 高速化 LoRA の強さ。1.0 推奨 |
55
+
56
+ 出力の `model`・`sampler`・`sigmas` は必ずセットで `BasicGuider` と `SamplerCustomAdvanced` に渡してください。
57
+
58
+ ## 注意
59
+
60
+ - **CFG とネガティブプロンプトは使いません。** `BasicGuider` を使ってください。`CFGGuider` や `KSampler` で CFG を 1 より大きくすると、計算が倍になるうえ絵が崩れます。
61
+ - **`ModelSamplingSD3`(shift)と `SkipLayerGuidanceDiTSimple`(layer 22 skip)は入れないでください。** 通常の 50 ステップ生成用の設定です。
62
+ - **解像度:** `EmptyFlux2LatentImage` で 16 の倍数、約 100 万画素がおすすめです。学習した解像度は 1024×1024、896×1152、832×1216、768×1344 と、それぞれの横向きです。
63
+ - **`weight_dtype`:** `default`(bf16)か `fp32` にしてください。fp8 には対応していません(エラーになります)。
64
+ - **仕組み:** LoRA は重みにマージせず fp32 で足し、time path も fp32 に置き換えます。差分が bf16 の丸め幅より小さく、マージすると効果が消えるためです。このぶん VRAM を約 1.5 GB 余分に使います。
65
+ - **seed:** `RandomNoise` の seed は、HF の `inference.py --seed` と同じ初期ノイズになります(どちらも CPU の `torch.randn`)。
66
+ - **他の LoRA との併用:** 画風 LoRA などは `LoraLoaderModelOnly` で重ねられます。このノードの前でも後でも構いません。ただし 4 ステップ化した状態での効き方は検証していません。
67
+
68
+ ## 動作確認(2026-10-17)
69
+
70
+ - **環境:** ComfyUI master(fc04da9、2026-10-16)、MOSS-Image 公式ノード(c1355e6)、torch 2.11 + CUDA 12.8、RTX PRO 6000。
71
+ - **結果:** 学習コードで作った正解画像(同じ seed 1234)とほぼ同じ絵になりました。テキストエンコーダの実装の違いによる細かい差だけです。
72
+ - **lodestones 8 ステップ:** 同じノードで動きました。
73
+ - **時間:** 1024² 1 枚あたり 4.5〜6.9 秒(テキストエンコード・VAE 込み、他のジョブと GPU を共用した状態)。
74
+
75
+ ![ComfyUI check](https://huggingface.co/darask0/rapid-moss/resolve/main/images/comfyui-check.jpg)
76
+
77
+ ## English
78
+
79
+ Custom node for 4-step, CFG-free MOSS-Image-SFT with the rapid-moss adapter (also runs lodestones' 8-step DMD). Install the
80
+ official MOSS-Image nodes, copy this folder to `ComfyUI/custom_nodes/rapid-moss/`, and put `lora.safetensors`,
81
+ `time_embedding.safetensors`, `dataspace.safetensors` in `ComfyUI/models/rapid_moss/<name>/`. Load
82
+ `example_workflows/rapid-moss-4step.json`, set the HF loader's `model_path` to MOSS-Image-SFT, and run. Use `BasicGuider`
83
+ (no CFG / negative), no `ModelSamplingSD3` or SLG, `weight_dtype` default/bf16/fp32 (fp8 unsupported). The node applies the
84
+ LoRA unmerged in fp32 and swaps in the fp32 time path (~1.5 GB extra VRAM), and returns the matching SAMPLER and SIGMAS for
85
+ `SamplerCustomAdvanced`.
comfyui/rapid-moss/__init__.py ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ from .nodes import NODE_CLASS_MAPPINGS, NODE_DISPLAY_NAME_MAPPINGS
2
+
3
+ __all__ = ['NODE_CLASS_MAPPINGS', 'NODE_DISPLAY_NAME_MAPPINGS']
comfyui/rapid-moss/example_workflows/rapid-moss-4step-api.json ADDED
@@ -0,0 +1,17 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "1": {"class_type": "MOSSImageHFLoader", "_meta": {"title": "MOSS-Image HF Loader (SFT)"},
3
+ "inputs": {"model_path": "/path/to/MOSS-Image-SFT", "weight_dtype": "default", "text_device": "default"}},
4
+ "2": {"class_type": "RapidMOSSAdapter", "_meta": {"title": "MOSS-Image Few-Step Adapter (rapid-moss)"},
5
+ "inputs": {"model": ["1", 0], "adapter": "rapid-moss-4step", "steps": 4, "lora_strength": 1.0}},
6
+ "3": {"class_type": "CLIPTextEncode", "_meta": {"title": "Prompt"},
7
+ "inputs": {"clip": ["1", 1], "text": "A high-quality anime illustration of a girl with long silver hair and teal eyes, wearing a yellow raincoat, standing at a rainy bus stop at night, neon reflections on wet asphalt"}},
8
+ "4": {"class_type": "BasicGuider", "_meta": {"title": "BasicGuider (no CFG)"},
9
+ "inputs": {"model": ["2", 0], "conditioning": ["3", 0]}},
10
+ "5": {"class_type": "RandomNoise", "_meta": {"title": "RandomNoise"}, "inputs": {"noise_seed": 42}},
11
+ "6": {"class_type": "EmptyFlux2LatentImage", "_meta": {"title": "Empty Flux 2 Latent"},
12
+ "inputs": {"width": 1024, "height": 1024, "batch_size": 1}},
13
+ "7": {"class_type": "SamplerCustomAdvanced", "_meta": {"title": "SamplerCustomAdvanced"},
14
+ "inputs": {"noise": ["5", 0], "guider": ["4", 0], "sampler": ["2", 1], "sigmas": ["2", 2], "latent_image": ["6", 0]}},
15
+ "8": {"class_type": "VAEDecode", "_meta": {"title": "VAE Decode"}, "inputs": {"samples": ["7", 0], "vae": ["1", 2]}},
16
+ "9": {"class_type": "SaveImage", "_meta": {"title": "Save Image"}, "inputs": {"filename_prefix": "rapid-moss", "images": ["8", 0]}}
17
+ }
comfyui/rapid-moss/example_workflows/rapid-moss-4step.json ADDED
@@ -0,0 +1,495 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "last_node_id": 9,
3
+ "last_link_id": 12,
4
+ "nodes": [
5
+ {
6
+ "id": 1,
7
+ "type": "MOSSImageHFLoader",
8
+ "pos": [
9
+ 40,
10
+ 120
11
+ ],
12
+ "size": [
13
+ 380,
14
+ 130
15
+ ],
16
+ "flags": {},
17
+ "order": 0,
18
+ "mode": 0,
19
+ "inputs": [],
20
+ "outputs": [
21
+ {
22
+ "name": "MODEL",
23
+ "type": "MODEL",
24
+ "links": [
25
+ 1
26
+ ],
27
+ "slot_index": 0
28
+ },
29
+ {
30
+ "name": "CLIP",
31
+ "type": "CLIP",
32
+ "links": [
33
+ 2
34
+ ],
35
+ "slot_index": 1
36
+ },
37
+ {
38
+ "name": "VAE",
39
+ "type": "VAE",
40
+ "links": [
41
+ 11
42
+ ],
43
+ "slot_index": 2
44
+ }
45
+ ],
46
+ "properties": {
47
+ "Node name for S&R": "MOSSImageHFLoader"
48
+ },
49
+ "widgets_values": [
50
+ "/path/to/MOSS-Image-SFT",
51
+ "default",
52
+ "default"
53
+ ],
54
+ "title": "MOSS-Image HF Loader (SFT)"
55
+ },
56
+ {
57
+ "id": 2,
58
+ "type": "RapidMOSSAdapter",
59
+ "pos": [
60
+ 470,
61
+ 60
62
+ ],
63
+ "size": [
64
+ 380,
65
+ 130
66
+ ],
67
+ "flags": {},
68
+ "order": 1,
69
+ "mode": 0,
70
+ "inputs": [
71
+ {
72
+ "name": "model",
73
+ "type": "MODEL",
74
+ "link": 1
75
+ }
76
+ ],
77
+ "outputs": [
78
+ {
79
+ "name": "model",
80
+ "type": "MODEL",
81
+ "links": [
82
+ 3
83
+ ],
84
+ "slot_index": 0
85
+ },
86
+ {
87
+ "name": "sampler",
88
+ "type": "SAMPLER",
89
+ "links": [
90
+ 7
91
+ ],
92
+ "slot_index": 1
93
+ },
94
+ {
95
+ "name": "sigmas",
96
+ "type": "SIGMAS",
97
+ "links": [
98
+ 8
99
+ ],
100
+ "slot_index": 2
101
+ }
102
+ ],
103
+ "properties": {
104
+ "Node name for S&R": "RapidMOSSAdapter"
105
+ },
106
+ "widgets_values": [
107
+ "rapid-moss-4step",
108
+ 4,
109
+ 1.0
110
+ ]
111
+ },
112
+ {
113
+ "id": 3,
114
+ "type": "CLIPTextEncode",
115
+ "pos": [
116
+ 470,
117
+ 250
118
+ ],
119
+ "size": [
120
+ 420,
121
+ 200
122
+ ],
123
+ "flags": {},
124
+ "order": 2,
125
+ "mode": 0,
126
+ "inputs": [
127
+ {
128
+ "name": "clip",
129
+ "type": "CLIP",
130
+ "link": 2
131
+ }
132
+ ],
133
+ "outputs": [
134
+ {
135
+ "name": "CONDITIONING",
136
+ "type": "CONDITIONING",
137
+ "links": [
138
+ 4
139
+ ],
140
+ "slot_index": 0
141
+ }
142
+ ],
143
+ "properties": {
144
+ "Node name for S&R": "CLIPTextEncode"
145
+ },
146
+ "widgets_values": [
147
+ "A high-quality anime illustration of a girl with long silver hair and teal eyes, wearing a yellow raincoat, standing at a rainy bus stop at night, neon reflections on wet asphalt"
148
+ ],
149
+ "title": "Prompt"
150
+ },
151
+ {
152
+ "id": 4,
153
+ "type": "BasicGuider",
154
+ "pos": [
155
+ 940,
156
+ 60
157
+ ],
158
+ "size": [
159
+ 260,
160
+ 50
161
+ ],
162
+ "flags": {},
163
+ "order": 3,
164
+ "mode": 0,
165
+ "inputs": [
166
+ {
167
+ "name": "model",
168
+ "type": "MODEL",
169
+ "link": 3
170
+ },
171
+ {
172
+ "name": "conditioning",
173
+ "type": "CONDITIONING",
174
+ "link": 4
175
+ }
176
+ ],
177
+ "outputs": [
178
+ {
179
+ "name": "GUIDER",
180
+ "type": "GUIDER",
181
+ "links": [
182
+ 6
183
+ ],
184
+ "slot_index": 0
185
+ }
186
+ ],
187
+ "properties": {
188
+ "Node name for S&R": "BasicGuider"
189
+ },
190
+ "widgets_values": [],
191
+ "title": "BasicGuider (no CFG)"
192
+ },
193
+ {
194
+ "id": 5,
195
+ "type": "RandomNoise",
196
+ "pos": [
197
+ 940,
198
+ 160
199
+ ],
200
+ "size": [
201
+ 260,
202
+ 82
203
+ ],
204
+ "flags": {},
205
+ "order": 4,
206
+ "mode": 0,
207
+ "inputs": [],
208
+ "outputs": [
209
+ {
210
+ "name": "NOISE",
211
+ "type": "NOISE",
212
+ "links": [
213
+ 5
214
+ ],
215
+ "slot_index": 0
216
+ }
217
+ ],
218
+ "properties": {
219
+ "Node name for S&R": "RandomNoise"
220
+ },
221
+ "widgets_values": [
222
+ 42,
223
+ "randomize"
224
+ ]
225
+ },
226
+ {
227
+ "id": 6,
228
+ "type": "EmptyFlux2LatentImage",
229
+ "pos": [
230
+ 940,
231
+ 290
232
+ ],
233
+ "size": [
234
+ 260,
235
+ 106
236
+ ],
237
+ "flags": {},
238
+ "order": 5,
239
+ "mode": 0,
240
+ "inputs": [],
241
+ "outputs": [
242
+ {
243
+ "name": "LATENT",
244
+ "type": "LATENT",
245
+ "links": [
246
+ 9
247
+ ],
248
+ "slot_index": 0
249
+ }
250
+ ],
251
+ "properties": {
252
+ "Node name for S&R": "EmptyFlux2LatentImage"
253
+ },
254
+ "widgets_values": [
255
+ 1024,
256
+ 1024,
257
+ 1
258
+ ]
259
+ },
260
+ {
261
+ "id": 7,
262
+ "type": "SamplerCustomAdvanced",
263
+ "pos": [
264
+ 1250,
265
+ 60
266
+ ],
267
+ "size": [
268
+ 300,
269
+ 110
270
+ ],
271
+ "flags": {},
272
+ "order": 6,
273
+ "mode": 0,
274
+ "inputs": [
275
+ {
276
+ "name": "noise",
277
+ "type": "NOISE",
278
+ "link": 5
279
+ },
280
+ {
281
+ "name": "guider",
282
+ "type": "GUIDER",
283
+ "link": 6
284
+ },
285
+ {
286
+ "name": "sampler",
287
+ "type": "SAMPLER",
288
+ "link": 7
289
+ },
290
+ {
291
+ "name": "sigmas",
292
+ "type": "SIGMAS",
293
+ "link": 8
294
+ },
295
+ {
296
+ "name": "latent_image",
297
+ "type": "LATENT",
298
+ "link": 9
299
+ }
300
+ ],
301
+ "outputs": [
302
+ {
303
+ "name": "output",
304
+ "type": "LATENT",
305
+ "links": [
306
+ 10
307
+ ],
308
+ "slot_index": 0
309
+ },
310
+ {
311
+ "name": "denoised_output",
312
+ "type": "LATENT",
313
+ "links": [],
314
+ "slot_index": 1
315
+ }
316
+ ],
317
+ "properties": {
318
+ "Node name for S&R": "SamplerCustomAdvanced"
319
+ },
320
+ "widgets_values": []
321
+ },
322
+ {
323
+ "id": 8,
324
+ "type": "VAEDecode",
325
+ "pos": [
326
+ 1250,
327
+ 230
328
+ ],
329
+ "size": [
330
+ 210,
331
+ 46
332
+ ],
333
+ "flags": {},
334
+ "order": 7,
335
+ "mode": 0,
336
+ "inputs": [
337
+ {
338
+ "name": "samples",
339
+ "type": "LATENT",
340
+ "link": 10
341
+ },
342
+ {
343
+ "name": "vae",
344
+ "type": "VAE",
345
+ "link": 11
346
+ }
347
+ ],
348
+ "outputs": [
349
+ {
350
+ "name": "IMAGE",
351
+ "type": "IMAGE",
352
+ "links": [
353
+ 12
354
+ ],
355
+ "slot_index": 0
356
+ }
357
+ ],
358
+ "properties": {
359
+ "Node name for S&R": "VAEDecode"
360
+ },
361
+ "widgets_values": []
362
+ },
363
+ {
364
+ "id": 9,
365
+ "type": "SaveImage",
366
+ "pos": [
367
+ 1600,
368
+ 60
369
+ ],
370
+ "size": [
371
+ 520,
372
+ 560
373
+ ],
374
+ "flags": {},
375
+ "order": 8,
376
+ "mode": 0,
377
+ "inputs": [
378
+ {
379
+ "name": "images",
380
+ "type": "IMAGE",
381
+ "link": 12
382
+ }
383
+ ],
384
+ "outputs": [],
385
+ "properties": {
386
+ "Node name for S&R": "SaveImage"
387
+ },
388
+ "widgets_values": [
389
+ "rapid-moss"
390
+ ]
391
+ }
392
+ ],
393
+ "links": [
394
+ [
395
+ 1,
396
+ 1,
397
+ 0,
398
+ 2,
399
+ 0,
400
+ "MODEL"
401
+ ],
402
+ [
403
+ 2,
404
+ 1,
405
+ 1,
406
+ 3,
407
+ 0,
408
+ "CLIP"
409
+ ],
410
+ [
411
+ 3,
412
+ 2,
413
+ 0,
414
+ 4,
415
+ 0,
416
+ "MODEL"
417
+ ],
418
+ [
419
+ 4,
420
+ 3,
421
+ 0,
422
+ 4,
423
+ 1,
424
+ "CONDITIONING"
425
+ ],
426
+ [
427
+ 5,
428
+ 5,
429
+ 0,
430
+ 7,
431
+ 0,
432
+ "NOISE"
433
+ ],
434
+ [
435
+ 6,
436
+ 4,
437
+ 0,
438
+ 7,
439
+ 1,
440
+ "GUIDER"
441
+ ],
442
+ [
443
+ 7,
444
+ 2,
445
+ 1,
446
+ 7,
447
+ 2,
448
+ "SAMPLER"
449
+ ],
450
+ [
451
+ 8,
452
+ 2,
453
+ 2,
454
+ 7,
455
+ 3,
456
+ "SIGMAS"
457
+ ],
458
+ [
459
+ 9,
460
+ 6,
461
+ 0,
462
+ 7,
463
+ 4,
464
+ "LATENT"
465
+ ],
466
+ [
467
+ 10,
468
+ 7,
469
+ 0,
470
+ 8,
471
+ 0,
472
+ "LATENT"
473
+ ],
474
+ [
475
+ 11,
476
+ 1,
477
+ 2,
478
+ 8,
479
+ 1,
480
+ "VAE"
481
+ ],
482
+ [
483
+ 12,
484
+ 8,
485
+ 0,
486
+ 9,
487
+ 0,
488
+ "IMAGE"
489
+ ]
490
+ ],
491
+ "groups": [],
492
+ "config": {},
493
+ "extra": {},
494
+ "version": 0.4
495
+ }
comfyui/rapid-moss/nodes.py ADDED
@@ -0,0 +1,149 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Few-step MOSS-Image adapters (rapid-moss DMAD 4-step, lodestones 8-step DMD) for ComfyUI.
2
+
3
+ Works on the MODEL from the MOSS-Image nodes (github.com/OpenMOSS/MOSS-Image, comfyui/moss-image).
4
+ An adapter is a folder in models/rapid_moss/ holding lora.safetensors, time_embedding.safetensors and
5
+ dataspace.safetensors. The node returns a patched MODEL plus the data-space SAMPLER and SIGMAS for
6
+ SamplerCustomAdvanced (use BasicGuider: no CFG, no negative prompt).
7
+ """
8
+ import os
9
+
10
+ import comfy.samplers
11
+ import folder_paths
12
+ import torch
13
+ from safetensors.torch import load_file
14
+ from torch.nn import functional as F
15
+
16
+ FOLDER = 'rapid_moss'
17
+ folder_paths.add_model_folder_path(FOLDER, os.path.join(folder_paths.models_dir, FOLDER))
18
+ FILES = ('lora.safetensors', 'time_embedding.safetensors', 'dataspace.safetensors')
19
+ PROJ = ('q', 'k', 'v', 'o', 'ffn_up_gate', 'ffn_down')
20
+ DM = 'diffusion_model.'
21
+
22
+
23
+ def _adapter_dirs():
24
+ found = {}
25
+ for root in folder_paths.get_folder_paths(FOLDER):
26
+ if os.path.isdir(root):
27
+ for name in sorted(os.listdir(root)):
28
+ path = os.path.join(root, name)
29
+ if name not in found and all(os.path.isfile(os.path.join(path, f)) for f in FILES):
30
+ found[name] = path
31
+ return found
32
+
33
+
34
+ class _DeviceCopy:
35
+ """fp32 tensors kept on the CPU, copied once to the device that first needs them."""
36
+
37
+ def __init__(self, *tensors):
38
+ self.cpu = tensors
39
+ self.device = None
40
+ self.copies = None
41
+
42
+ def get(self, device):
43
+ if device != self.device:
44
+ self.copies = [t.to(device) for t in self.cpu]
45
+ self.device = device
46
+ return self.copies
47
+
48
+
49
+ class _FP32Linear(torch.nn.Module):
50
+ """fp32 linear outside ComfyUI's weight management. `weight` is a plain attribute so the MOSS
51
+ forward (which casts its input to `.weight.dtype`) runs this path in fp32."""
52
+
53
+ def __init__(self, weight, bias):
54
+ super().__init__()
55
+ self.weight = weight
56
+ self.tensors = _DeviceCopy(weight, bias)
57
+
58
+ def forward(self, x):
59
+ weight, bias = self.tensors.get(x.device)
60
+ return F.linear(x.float(), weight, bias)
61
+
62
+
63
+ def _lora_forward(module, down, up, scale):
64
+ """Unmerged fp32 LoRA: the delta is below bf16 rounding, so it must not be merged into the weight."""
65
+ class_forward = type(module).forward
66
+ tensors = _DeviceCopy(down, up)
67
+
68
+ def forward(x, *args, **kwargs):
69
+ out = class_forward(module, x, *args, **kwargs)
70
+ d, u = tensors.get(x.device)
71
+ return out + (((x.float() @ d.t()) @ u.t()) * scale).to(out.dtype)
72
+ return forward
73
+
74
+
75
+ @torch.no_grad()
76
+ def sample_dataspace(model, x, sigmas, extra_args=None, callback=None, disable=None, Q=None, codes=None):
77
+ """x0 = 0; for r: x = z + Q x0, v = net(x, code_r), x0 = z - v (one noise z, no schedule, no CFG)."""
78
+ extra_args = {} if extra_args is None else extra_args
79
+ z = x.float() # sigmas[0] == 1, so the sampler receives pure noise
80
+ mix = Q.to(z.device, torch.float32)
81
+ x0 = torch.zeros_like(z)
82
+ s_in = z.new_ones([z.shape[0]])
83
+ for r in range(len(sigmas) - 1):
84
+ x_in = z + torch.einsum('oc,bchw->bohw', mix, x0)
85
+ s = float(codes[r]) / 1000.0 # flow model: timestep = sigma * 1000 = code
86
+ denoised = model(x_in, s * s_in, **extra_args).float() # = x_in - s * v
87
+ x0 = z - (x_in - denoised) / s
88
+ if callback is not None:
89
+ callback({'x': x_in, 'i': r, 'sigma': sigmas[r], 'sigma_hat': sigmas[r], 'denoised': x0})
90
+ return x0
91
+
92
+
93
+ class RapidMOSSAdapter:
94
+ @classmethod
95
+ def INPUT_TYPES(cls):
96
+ names = list(_adapter_dirs()) or ['(no adapter folder in models/rapid_moss)']
97
+ return {'required': {
98
+ 'model': ('MODEL',),
99
+ 'adapter': (names,),
100
+ 'steps': ('INT', {'default': 4, 'min': 1, 'max': 8,
101
+ 'tooltip': 'Network evaluations; up to the adapter step-code count (4 for rapid-moss, 8 for lodestones).'}),
102
+ 'lora_strength': ('FLOAT', {'default': 1.0, 'min': 0.0, 'max': 2.0, 'step': 0.05}),
103
+ }}
104
+
105
+ RETURN_TYPES = ('MODEL', 'SAMPLER', 'SIGMAS')
106
+ RETURN_NAMES = ('model', 'sampler', 'sigmas')
107
+ FUNCTION = 'apply'
108
+ CATEGORY = 'MOSS-Image/rapid-moss'
109
+ DESCRIPTION = ('Few-step MOSS-Image: patches the MOSS-Image MODEL with a data-space adapter (unmerged fp32 LoRA, '
110
+ 'fp32 time path) and returns the matching SAMPLER and SIGMAS for SamplerCustomAdvanced + BasicGuider.')
111
+
112
+ def apply(self, model, adapter, steps, lora_strength):
113
+ dirs = _adapter_dirs()
114
+ if adapter not in dirs:
115
+ raise FileNotFoundError(f'Put {", ".join(FILES)} in a folder under models/{FOLDER}/ (got {adapter!r}).')
116
+ path = dirs[adapter]
117
+ lora = load_file(os.path.join(path, 'lora.safetensors'))
118
+ time_path = load_file(os.path.join(path, 'time_embedding.safetensors'))
119
+ data = load_file(os.path.join(path, 'dataspace.safetensors'))
120
+ codes = data['step_codes'].float()
121
+ if steps > len(codes):
122
+ raise ValueError(f'{adapter} has {len(codes)} step codes; steps must be <= {len(codes)}.')
123
+
124
+ patched = model.clone()
125
+ base = patched.get_model_object(DM + 'shared_block_modulation')
126
+ if base.weight.dtype not in (torch.bfloat16, torch.float16, torch.float32):
127
+ raise ValueError('Load MOSS-Image with weight_dtype default/bf16/fp32; fp8 weights are not supported here.')
128
+ patched.add_object_patch(DM + 'time_embedding', torch.nn.Sequential(
129
+ _FP32Linear(time_path['time_embedding.0.weight'].float(), time_path['time_embedding.0.bias'].float()),
130
+ torch.nn.SiLU(),
131
+ _FP32Linear(time_path['time_embedding.2.weight'].float(), time_path['time_embedding.2.bias'].float())))
132
+ for name in ('shared_block_modulation', 'video_head_modulation'):
133
+ weight = patched.get_model_object(DM + name).weight.detach().to('cpu', torch.float32)
134
+ patched.add_object_patch(DM + name, _FP32Linear(weight, time_path[name + '.bias'].float()))
135
+ for i in range(len(patched.get_model_object(DM + 'layers'))):
136
+ for proj in PROJ:
137
+ target = f'{DM}layers.{i}.proj.{proj}'
138
+ down = lora[target + '.lora_down.weight'].float()
139
+ up = lora[target + '.lora_up.weight'].float()
140
+ scale = lora_strength * float(lora[target + '.alpha']) / down.shape[0]
141
+ patched.add_object_patch(target + '.forward', _lora_forward(patched.get_model_object(target), down, up, scale))
142
+
143
+ sampler = comfy.samplers.KSAMPLER(sample_dataspace, extra_options={'Q': data['latent_mix.Q'].float(), 'codes': codes})
144
+ sigmas = torch.cat((codes[:steps] / 1000.0, torch.zeros(1)))
145
+ return (patched, sampler, sigmas)
146
+
147
+
148
+ NODE_CLASS_MAPPINGS = {'RapidMOSSAdapter': RapidMOSSAdapter}
149
+ NODE_DISPLAY_NAME_MAPPINGS = {'RapidMOSSAdapter': 'MOSS-Image Few-Step Adapter (rapid-moss)'}
images/comfyui-check.jpg ADDED

Git LFS Details

  • SHA256: 50e5e8118c540ca79e0d9a30071f4568066174e138e834449761bb3ed66bd240
  • Pointer size: 131 Bytes
  • Size of remote file: 742 kB