# Install and run GLM5.3 Flash CIRU STRIX IU4 This build runs across **two 128-GiB AMD Strix Halo systems** connected by USB4. Use the packaged runtime: a stock vLLM or Transformers installation does not include this model's weight loader, gfx1151 kernels, or DFlash2 path. The default context is 128K; a 256K profile is also available. The production transport uses **NHI (Native Host Interface)**, the USB4 controller's direct data-transfer interface, through `/dev/tbstream0`. This avoids the network stack for the model's M8 all-reduce. CiruStrixLink handles link setup and endpoint preparation; it is recommended but not mandatory. | Setup | Who manages the link? | Model service | |---|---|---| | [StrixLink + NHI, recommended](#recommended-strixlink-and-nhi) | CiruStrixLink | Packaged privileged system service | | [Manual NHI](#manual-nhi) | Your existing USB4/NHI setup | Your capability-bearing service/launcher | | [Portable compatibility](#portable-compatibility-and-user-services) | StrixLink or your existing USB4 network | Ordinary user systemd service or foreground launcher | All three use the same model and [mirrored generation frontend](#required-mirrored-generation-frontend). Portable sockets are a compatibility option, not the NHI performance path. ## Requirements - Two gfx1151 Strix Halo hosts with 128 GiB unified memory each; one model request at a time is the supported profile. - A direct USB4 connection and compatible kernel/device support. The tested system is NixOS with Linux 7.2.2 and ROCm 10. Other distributions are not claimed as tested deployment targets; the runtime wheels require Ubuntu 24.04+ or an equivalently compatible x86-64/glibc environment. - Python 3 and venv support for the download CLI, plus `uv`, the host build prerequisites, and the pinned runtime installed below. - Local NVMe for each rank's approximately 83.25 GiB target weights, the separate draft/runtime, and substantial additional [prefix-cache space](#persistent-disk-prefix-cache). - Enough free unified memory for the target, draft, 12 GiB GPU KV cache and 8 GiB host prefix staging per rank. Stop competing models as needed; this configuration does not guarantee 20 GB free per host. The production launcher requires the separately downloaded DFlash2 draft. Its CC BY-NC-ND 4.0 license is independent of the target model; commercial use requires suitable rights. See [component licenses](../THIRD_PARTY_NOTICES.md). Throughout this guide, replace `PEER_USB4_ADDRESS`, `RANK_0_USB4_ADDRESS`, `THIS_HOST_USB4_ADDRESS`, and `FRONTEND_HOST` with your addresses. Each host has a different `NODE_RANK` (`0` or `1`) and `HOST_IP`; both must use the same `MASTER_ADDR`, `MASTER_PORT`, `EPOCH`, context profile, and production settings. ## Download the model and draft Run on each host. Download only that host's rank directory: ```bash sudo install -d -o "$USER" -g "$(id -gn)" \ /srv/llm/models /srv/llm/runtime /srv/llm/venvs /srv/llm/cache HF_ENV="${XDG_DATA_HOME:-$HOME/.local/share}/ciru-hf" if [[ ! -x "$HF_ENV/bin/hf" ]]; then python3 -m venv "$HF_ENV" "$HF_ENV/bin/pip" install -U huggingface_hub fi HF="$HF_ENV/bin/hf" MODEL_REPO=jcbtc/GLM5.3-Flash-CIRU-STRIX-IU4 MODEL_ROOT=/srv/llm/models/GLM5.3-Flash-CIRU-STRIX-IU4 NODE_RANK=0 # use 1 on the peer "$HF" download "$MODEL_REPO" \ --include "config/*" \ --include "docs/*" \ --include "runtime/**" \ --include "rank-${NODE_RANK}/*" \ --include "systemd/*" \ --include "context-profiles/*" \ --include "tools/*" \ --include "LICENSE" \ --include "LICENSE_SCOPE.md" \ --include "THIRD_PARTY_NOTICES.md" \ --include "launch-node.sh" \ --include "run-node.sh" \ --include "install-user-service.sh" \ --include "install-generation-frontend-user-service.sh" \ --include "install-nhi-system-service.sh" \ --include "select-context-profile.sh" \ --include "select-nhi-context-profile.sh" \ --local-dir "$MODEL_ROOT" DFLASH_REVISION=bf582e4eacc1810f76656d1811693ff6c6737d2a "$HF" download incoai/GLM-5.3-Flash-DFlash2 \ --revision "$DFLASH_REVISION" \ --local-dir /srv/llm/models/GLM-5.3-Flash-DFlash2 ``` Keep the packaged rank manifests and shard filenames unchanged. The launcher loads rank-local runtime-prepacked weights rather than re-quantizing the model. ## Install the runtime The self-contained bundle installs the pinned Python 3.14.3, ROCm 10/PyTorch, vLLM, AITER, TileLang, and source trees. Exact artifact versions, compatible host prerequisites, and the detailed install recipe are in [runtime/packages/README.md](../runtime/packages/README.md). On NixOS, import the included module in your host configuration and apply it once on each machine: ```nix { imports = [ /path/to/GLM5.3-Flash-CIRU-STRIX-IU4/runtime/packages/nixos-module.nix ]; } ``` ```bash sudo nixos-rebuild switch cd "$MODEL_ROOT/runtime/packages" nix-shell ./shell.nix --run 'bash ./INSTALL-RUNTIME.sh' ``` The module enables `nix-ld`; the shell supplies host compiler wrappers and development headers. On a compatible non-NixOS host, install the prerequisites from the runtime README and run `bash ./INSTALL-RUNTIME.sh` directly. The default installation paths are: ```text /srv/llm/runtime/vllm-glm53-strix /srv/llm/runtime/aiter-gfx1151 /srv/llm/venvs/vllm-rocm10-gfx1151 /srv/llm/runtime/vllm-glm53-strix/runtime-env.sh ``` NHI additionally requires the [root-owned runtime staging](#privileged-direct-nhi-system-service) described below. Do not replace the bundled runtime with a stock PyPI vLLM wheel or omit TileLang; the packaged dependencies are part of the serving path. ## Recommended: StrixLink and NHI The direct GPU path needs the [qualified Linux 7.2.2 patch series](https://github.com/ciru-ai/CiruStrixLink/tree/master/kernel/linux-7.2.2), including the DMA-BUF import ABI and MSI-X reliability fix. A stock kernel version alone is insufficient. [Transport details and microsecond measurements](STRIXLINK-TRANSPORT.md) explain what each component provides. ### Install StrixLink The model repository includes CiruStrixLink 0.3.3; no separate GitHub checkout is needed. On both hosts: ```bash VERSION=0.3.3 mkdir -p /tmp/ciru-strixlink-release "$HF" download "$MODEL_REPO" \ --include "tools/ciru-strixlink-${VERSION}-linux-amd64.tar.gz" \ "tools/ciru-strixlink-${VERSION}-SHA256SUMS" \ --local-dir /tmp/ciru-strixlink-model-release (cd /tmp/ciru-strixlink-model-release/tools && \ sha256sum -c "ciru-strixlink-${VERSION}-SHA256SUMS") tar -xzf \ "/tmp/ciru-strixlink-model-release/tools/ciru-strixlink-${VERSION}-linux-amd64.tar.gz" \ -C /tmp/ciru-strixlink-release sudo install -m 0755 \ /tmp/ciru-strixlink-release/ciru-strixlink \ /usr/local/bin/ciru-strixlink ciru-strixlink prerequisites ``` Connect the cable. Use role `a` on one host and role `b` on the other. Preview the proposed configuration, then apply it: ```bash # First host: ciru-strixlink setup --role a sudo ciru-strixlink setup --role a --apply ``` ```bash # Second host: ciru-strixlink setup --role b sudo ciru-strixlink setup --role b --apply ``` Check the connection: ```bash ciru-strixlink doctor --peer PEER_USB4_ADDRESS # Run on one host: ciru-strixlink serve ``` ```bash # Run on the other: ciru-strixlink test \ --peer PEER_USB4_ADDRESS \ --duration 7s \ --streams 4 \ --output ciru-strixlink-report.json \ --env-file ciru-strixlink.env ``` ### Optional paired Launch page StrixLink 0.3.3 can coordinate this fixed two-rank model from its third browser tab. Model control is disabled by default. It requires a mode-0600 token file with the same value on both hosts, complementary fixed ranks, fixed USB4 IPv4 peers, a model frontend URL, a loopback-only console, and the narrow packaged helper on each machine. On the peer model host: ```bash ciru-strixlink agent \ --token-file TOKEN_FILE \ --model-control \ --model-rank 1 \ --model-peer RANK_0_USB4_ADDRESS ``` On the console model host: ```bash ciru-strixlink ui \ --peer RANK_1_USB4_ADDRESS \ --token-file TOKEN_FILE \ --model-url http://127.0.0.1:8083 \ --model-control \ --model-rank 0 ``` From a desktop, use an authenticated tunnel and open the Launch tab: ```bash ssh -N -L 7749:127.0.0.1:7749 USER@CONSOLE_MODEL_HOST ``` Open `http://127.0.0.1:7749/#/launch`. The desktop does not run StrixLink or receive model-host privileges. The page shows exact copyable NixOS and generic Linux permission instructions when control is locked. See the [upstream browser-console guide](https://github.com/ciru-ai/CiruStrixLink#browser-console) for the complete security and sudoers contract. To change context, unload the pair, choose the same profile for both ranks in the Launch page, and load again. Context controls intentionally remain locked while the model is active. Loading prevents the portable and NHI forms of this GLM deployment from running together. Unrelated applications remain the operator’s responsibility. ### Prepare both NHI endpoints On each host, collect a status report: ```bash STRIX_CONFIG="${XDG_CONFIG_HOME:-$HOME/.config}/GLM5.3-Flash-CIRU-STRIX-IU4" mkdir -p "$STRIX_CONFIG" ciru-strixlink transport status \ --peer PEER_USB4_ADDRESS \ --output "$STRIX_CONFIG/rank-${NODE_RANK}.transport.json" ``` Bring both reports together on one host and reconcile them: ```bash ciru-strixlink transport reconcile \ --a "$STRIX_CONFIG/rank-0.transport.json" \ --b "$STRIX_CONFIG/rank-1.transport.json" \ --output "$STRIX_CONFIG/pair.transport.json" ``` If `arm_allowed=true`, preview and apply endpoint preparation on **both hosts concurrently**: ```bash ciru-strixlink transport endpoint prepare --peer PEER_USB4_ADDRESS sudo ciru-strixlink transport endpoint prepare \ --peer PEER_USB4_ADDRESS --apply ``` Collect new reports, exchange them, and reconcile the prepared pair: ```bash ciru-strixlink transport status \ --peer PEER_USB4_ADDRESS \ --output "$STRIX_CONFIG/rank-${NODE_RANK}.prepared.transport.json" ciru-strixlink transport reconcile \ --a "$STRIX_CONFIG/rank-0.prepared.transport.json" \ --b "$STRIX_CONFIG/rank-1.prepared.transport.json" \ --output "$STRIX_CONFIG/pair-prepared.transport.json" ``` Make that prepared-pair report available on both hosts. When it reports `nhi_ready=true` and `lease_available=true`, each host generates its own environment: ```bash ciru-strixlink transport env \ --peer PEER_USB4_ADDRESS \ --mode nhi \ --runtime vllm \ --pair-report "$STRIX_CONFIG/pair-prepared.transport.json" \ --output "$STRIX_CONFIG/ciru-strixlink-nhi.env" ``` If preparation fails or leaves partial state, use the two-sided [stop and cleanup sequence](#stopping-and-restarting) before trying again. The [StrixLink guide](./CiruStrixLink-GLM5.3-Flash-CIRU-STRIX-IU4.md) has additional lifecycle and browser-console instructions. ### Privileged direct-NHI system service NHI DMA-BUF import needs `CAP_SYS_RAWIO`, a powerful Linux capability. The packaged system unit grants it to a constrained service, not to the ordinary user unit. The service user must belong to `render` and `video`; on NixOS, declare both in `users.users..extraGroups` and apply the configuration. Before installing the unit, stage the venv, complete CPython runtime, vLLM source, AITER source, environment script, and gfx1151 libraries under a root-owned path such as `/opt/ciru/glm53-iu4`, matching [`systemd/nhi.env.example`](../systemd/nhi.env.example). The code must be owned by `root:root`, not group/world-writable, and readable/traversable by the service user. Use copies or reflinks, not hardlinks to user-owned code. A copied UV venv can still point into a user's home directory. Stage the complete Python 3.14.3 runtime at `/opt/ciru/glm53-iu4/python`, make `venv/bin/python` and its aliases resolve to `python/bin/python3.14`, and set `home = /opt/ciru/glm53-iu4/python/bin` in `venv/pyvenv.cfg`. The service installer checks the staging but does not perform that copy/relink. Follow the [runtime staging instructions](../runtime/packages/README.md#privileged-nhi-runtime-staging), including the NixOS `CC`/`CXX` systemd drop-in. Those values must use the Nix host compiler wrappers, not bare ROCm clang. Create a dedicated NHI node configuration on each host and edit its paths, rank, addresses, epoch, and local NVMe cache directory: ```bash cp "$MODEL_ROOT/systemd/nhi.env.example" "$STRIX_CONFIG/node-nhi.env" chmod 0600 "$STRIX_CONFIG/node-nhi.env" ``` Keep it separate from the portable user service's `node.env`. Install the unit with the dedicated node configuration and generated transport file: ```bash sudo bash "$MODEL_ROOT/install-nhi-system-service.sh" \ "$(id -un)" \ "$STRIX_CONFIG/node-nhi.env" \ "$STRIX_CONFIG/ciru-strixlink-nhi.env" ``` Installation reloads systemd but does not start or enable the model. Stop any portable instance on both hosts, choose the same context profile, then start the two NHI units together: ```bash systemctl --user stop GLM5.3-Flash-CIRU-STRIX-IU4.service sudo glm53-nhi-context "$(id -un)" 2 sudo systemctl start "GLM5.3-Flash-CIRU-STRIX-IU4-nhi@$(id -un).service" ``` Do not enable the model unit at boot: the pair's endpoint state must be reconciled before starting. If either rank fails, stop its peer as well. Once both rank APIs are ready, start the frontend below. ## Required mirrored generation frontend The model uses two independent rank launchers. Every generation request must reach both rank APIs concurrently with the same body. The packaged frontend does this for you, returning rank 0 and draining rank 1. **Send all generation to port 8083, never directly to one rank's port 8100.** Install the frontend user unit on either host: ```bash bash "$MODEL_ROOT/install-generation-frontend-user-service.sh" ``` Edit `~/.config/GLM5.3-Flash-CIRU-STRIX-IU4/frontend.env`: - `GLM_PRIMARY_URL`: rank 0 API, for example `http://RANK_0_HOST:8100`. - `GLM_MIRROR_URL`: rank 1 API, for example `http://RANK_1_HOST:8100`. - Keep the model ID `GLM5.3-Flash-CIRU-STRIX-IU4`. - The listener defaults to `127.0.0.1:8083`. To serve remote clients, bind `GLM_FRONTEND_HOST` to a private reachable address and put authentication and access control in front of it. After both model APIs are ready: ```bash systemctl --user start GLM5.3-Flash-CIRU-STRIX-IU4-frontend.service journalctl --user -u GLM5.3-Flash-CIRU-STRIX-IU4-frontend.service -f ``` The frontend has no StrixLink dependency. For a custom service manager: ```bash MODEL_ROOT=/srv/llm/models/GLM5.3-Flash-CIRU-STRIX-IU4 \ VLLM_VENV=/srv/llm/venvs/vllm-rocm10-gfx1151 \ GLM_PRIMARY_URL=http://RANK_0_HOST:8100 \ GLM_MIRROR_URL=http://RANK_1_HOST:8100 \ bash "$MODEL_ROOT/runtime/generation-frontend/run.sh" ``` The frontend mirrors `/v1/chat/completions`, `/v1/completions`, and `/v1/responses`. Use the chat endpoint for normal model interaction so the packaged GLM chat template and reasoning/tool behavior apply. Rank-local port 8100 is for health, metrics, tokenizer/render inspection, and debugging. ## Chat API ```bash curl http://FRONTEND_HOST:8083/v1/chat/completions \ -H 'Content-Type: application/json' \ -d '{ "model": "GLM5.3-Flash-CIRU-STRIX-IU4", "messages": [ {"role": "user", "content": "Write a robust Python LRU cache."} ], "temperature": 1.0, "top_p": 0.95, "reasoning_effort": "max", "chat_template_kwargs": {"clear_thinking": true}, "max_tokens": 8192 }' ``` `max_tokens` is the new-output allowance, including reasoning. Choose enough for the task while leaving room inside the total context limit; 8192 is only this example's allowance. Add `"stream": true` for streaming responses. For tools, pass OpenAI-compatible `tools` and `tool_choice` fields, including `"tool_choice": "auto"` for automatic selection. Execute permitted calls in your client, append the assistant tool call and matching tool-result messages, then send the conversation again. Do not manually insert GLM control tokens. Keep `clear_thinking=true` in normal multi-turn chat so previous hidden reasoning is not carried forward as conversation content. Apply appropriate tool permissions and confirmations in the client. ### Production settings The launcher already supplies the intended engine settings; retain them when integrating the model into another service manager: - vLLM V2, target TP2 with the external launcher, BF16 activations, and hybrid IU4 target weights. - DFlash2 k=5, probabilistic draft sampling, and standard rejection sampling. - The packaged chat template and generation config; temperature 1.0, top-p 0.95, reasoning effort `max`, and `clear_thinking=true` for chat requests. - AITER attention enabled, AITER MoE and skinny GEMM disabled, direct-route M1 MoE and the packaged tuned MoE configuration enabled. - One sequence, eager execution, asynchronous scheduling disabled, chunked prefill enabled, block size 128, and 2,304 maximum batched tokens. - Prefix caching disabled by default. The optional prompt-only disk tier uses 8 GiB host staging and one block per offload chunk, but requires an external disk quota and monitoring before production use. - `glm47` reasoning/tool parsers and the packaged M8 NHI adapter on a ready NHI pair. The 30,000 ms NHI timeout is a failure deadline; successful transfers return immediately. `MAX_NUM_BATCHED_TOKENS` is configurable, but larger values are experimental and are not automatically faster. The default also provides recurrent-state checkpoints needed by the hybrid prefix cache. ## Context profiles Fresh installs choose profile 2. Existing selections are preserved on reinstallation. Apply the same choice to both hosts: ```bash # NHI system service: sudo glm53-nhi-context "$(id -un)" list sudo glm53-nhi-context "$(id -un)" 2 # Portable user service: glm53-context list glm53-context 2 ``` | Number | Profile | Total request limit | GPU KV per rank | Host prefix staging | |---:|---|---:|---:|---:| | 1 | 64K, lower-memory fallback | 65,664 tokens | 6 GiB | 8 GiB | | 2 | 128K, default | 131,200 tokens | 12 GiB | 8 GiB | | 3 | 256K, experimental | 262,272 tokens | 8 GiB | 8 GiB | The limit includes prompt, generated reasoning, and answer tokens. The extra 128 tokens are scheduler/speculation headroom above the nominal context. The selectors update only next-start configuration; they do not start or restart the model. Stop both ranks, change both selections, then restart the pair with ready endpoints. No RoPE scaling is applied. Published performance tests cover the 128K profile, including a 131,000-token input and persistent cache reuse. The 256K/8 GiB profile has passed paired startup, exact-context serving validation, HumanEval 0–9, and a 65,680-token recovery stress probe. That long-context probe tests a minimum generation floor after a large prefill; it is not representative of normal short-prompt chat speed. Its initial depth comparison was not fully host-isolated and is retained only as directional evidence. This is not a complete 256K quality sweep. Available unified memory remains the limiting resource; a 16 GiB KV allocation did not fit the validated 128-GiB-per-host deployment. See the [recorded results](../benchmarks/README.md) for measurement scope. ## DFlash and cache controls The launcher reads two per-user files at startup. Keep them identical on both ranks and edit them only while the pair is stopped: ```bash mkdir -p ~/.config/ciru-glm53-iu4 printf '5\n' > ~/.config/ciru-glm53-iu4/dflash-tokens printf '0\n' > ~/.config/ciru-glm53-iu4/prefix-cache-enabled ``` `dflash-tokens` accepts 0 through 7. Zero omits DFlash entirely for a true target-only comparison. Invalid values stop startup; missing files use k=5 and cache-off. The Launch page reports a value as known only when both ranks report the same setting. ## Persistent disk prefix cache Prefix caching is disabled by default. When explicitly enabled, matching system prompts, tool schemas, and documents can be reused across requests and service restarts. Completed prompt blocks pass through an 8 GiB host-memory LRU and are written to local NVMe at `PREFIX_CACHE_ROOT`. Only prompt blocks are offloaded; generated tails are not written to this cache. Each TP rank keeps its own files. Cached KV blocks are not transferred over USB4. **8 GiB limits RAM staging, not disk storage.** The filesystem tier has no built-in byte quota or automatic eviction. The 256K investigation found retained cache trees of 219 GiB on rank 0 and 266 GiB on rank 1. Do not enable this tier on a general-purpose filesystem. Use a dedicated quota-managed filesystem, monitor free space and cache-write failures, and define cleanup behavior before opting in. Set `PREFIX_CACHE_ROOT` to a suitable directory on each host. Keeping that directory preserves the disk cache across service restarts. `PREFIX_CACHE_ENABLED=1` opts in; `0` is the production default. The NHI unit's private `/dev/shm` tmpfs is separate temporary staging scratch, capped at 16 GiB and released when the service stops. This does not delete the persistent NVMe cache. Both GPU KV and host staging consume unified memory; leave space for the OS and other software. ## Manual NHI You can use an existing direct-NHI setup without running the StrixLink app. Your service manager must supply a root-owned runtime and grant the model process `CAP_SYS_RAWIO`. Use the same model/runtime paths and production settings above, plus: ```bash export TRANSPORT_MODE=nhi export TRANSPORT_INTERFACE=thunderbolt0 export VLLM_NHI_DEVICE=/dev/tbstream0 export ALLOW_PRIVILEGED_NHI=1 ``` The operator owns endpoint readiness and matching epoch, HopID, ring, and peer state. `ALLOW_PRIVILEGED_NHI=1` acknowledges this mode; it does not grant the capability. The ordinary user service cannot run it. Launch a trusted, root-owned copy of `run-node.sh` and its companion scripts from your capability-bearing service with the per-rank values shown in the foreground example below. Do not mix an existing endpoint owner with StrixLink endpoint preparation. The packaged NHI service installer accepts a StrixLink-generated transport file. A fully caller-owned setup can instead retain its own service and transport lifecycle; the model does not require the StrixLink process at runtime. For a service that already sets all RCCL/Gloo and optional NHI variables, the launcher also offers external mode: ```bash export TRANSPORT_MODE=external export NCCL_SOCKET_IFNAME='=my-usb4-interface' export GLOO_SOCKET_IFNAME=my-usb4-interface export VLLM_NHI_M8_ALLREDUCE=0 # set 1 only for your prepared NHI path ``` External mode preserves caller-supplied transport values. An NHI-enabled external setup still requires the capability and two-sided endpoint handling. `MASTER_ADDR` must always be the reachable rank-0 address. ## Portable compatibility and user services For an existing USB4 network interface, the launcher can pin RCCL/Gloo to it without StrixLink: ```bash export TRANSPORT_MODE=portable export TRANSPORT_INTERFACE=thunderbolt0 ``` Alternatively, generate a portable environment with StrixLink after setting up and testing the USB4 network: ```bash STRIX_CONFIG="${XDG_CONFIG_HOME:-$HOME/.config}/GLM5.3-Flash-CIRU-STRIX-IU4" mkdir -p "$STRIX_CONFIG" ciru-strixlink transport env \ --peer PEER_USB4_ADDRESS \ --mode portable \ --runtime vllm \ --output "$STRIX_CONFIG/ciru-strixlink-portable.env" ``` ### User systemd service On each host: ```bash bash "$MODEL_ROOT/install-user-service.sh" ``` Edit `~/.config/GLM5.3-Flash-CIRU-STRIX-IU4/node.env`. Use `TRANSPORT_MODE=portable` with your interface, `strixlink` with the generated portable `STRIXLINK_ENV`, or an unprivileged caller-owned `external` configuration. See [`external-transport.env.example`](../systemd/external-transport.env.example). Set the per-rank paths/addresses and shared epoch described at the start of this guide. Installation reloads the user manager but does not start the model. When both configurations are ready, start both hosts together: ```bash systemctl --user start GLM5.3-Flash-CIRU-STRIX-IU4.service journalctl --user -u GLM5.3-Flash-CIRU-STRIX-IU4.service -f ``` Start the same [mirrored frontend](#required-mirrored-generation-frontend) after both rank APIs are ready. Do not run the portable and NHI model units simultaneously. The user unit is not enabled at boot and does not independently restart a failed rank; start/restart the pair together. ### Foreground launch example This example uses StrixLink's portable environment. For a manually configured USB4 network, replace `TRANSPORT_MODE` and `STRIXLINK_ENV` with the portable interface settings above. For manual NHI, use your capability-bearing service and the NHI settings instead. ```bash export MODEL_ROOT=/srv/llm/models/GLM5.3-Flash-CIRU-STRIX-IU4 export DFLASH_MODEL=/srv/llm/models/GLM-5.3-Flash-DFlash2 export VLLM_SOURCE=/srv/llm/runtime/vllm-glm53-strix export VLLM_VENV=/srv/llm/venvs/vllm-rocm10-gfx1151 export VLLM_RUNTIME_ENV=/srv/llm/runtime/vllm-glm53-strix/runtime-env.sh export AITER_SOURCE=/srv/llm/runtime/aiter-gfx1151 export TRANSPORT_MODE=strixlink export STRIXLINK_ENV="${XDG_CONFIG_HOME:-$HOME/.config}/GLM5.3-Flash-CIRU-STRIX-IU4/ciru-strixlink-portable.env" export MASTER_ADDR=RANK_0_USB4_ADDRESS export PREFIX_CACHE_ROOT=/path/on/local/nvme/GLM5.3-Flash-CIRU-STRIX-IU4 export NODE_RANK=0 # use 1 on the peer export HOST_IP=THIS_HOST_USB4_ADDRESS export EPOCH=1 # choose the same new value on both hosts export API_PORT=8100 bash "$MODEL_ROOT/run-node.sh" ``` For NHI, replace the `/srv/llm` runtime paths with your root-owned staging paths. `run-node.sh` enters the packaged `shell.nix` on NixOS when available; elsewhere it uses the installed host toolchain. ## Stopping and restarting Always stop and restart both model ranks as a pair. The frontend may remain installed, but do not submit generation while either rank is unavailable. For the packaged NHI service, stop both units before releasing endpoints: ```bash sudo systemctl stop "GLM5.3-Flash-CIRU-STRIX-IU4-nhi@$(id -un).service" ciru-strixlink transport endpoint cleanup sudo ciru-strixlink transport endpoint cleanup --apply ``` Preview cleanup on each host and apply it concurrently. Collect fresh status and reconcile again before preparing the next NHI start. If preparation failed partway through, clean both endpoints. Before switching to portable, stop both NHI units and confirm both endpoints have no remaining holder or partial state. The packaged units do not perform one-sided automatic cleanup, restart, or boot activation.