INT8 ConvRot変換ツールの解説

Model (net)
├─ t_embedder (2個)
├─ t_embedding_norm (1個)
├─ x_embedder (1個)
├─ blocks.0
│   ├─ adaln_modulation_cross_attn (2個)
│   ├─ adaln_modulation_mlp (2個)
│   ├─ adaln_modulation_self_attn (2個)
│   ├─ cross_attn (6個)
│   ├─ mlp (2個)
│   └─ self_attn (6個)
├─ blocks.1
├─ ...
├─ blocks.27
├─ final_layer (3個)
└─ llm_adapter (118個)

import sys
from safetensors import safe_open

def main():
    if len(sys.argv) != 2:
        print("Usage:")
        print(f"  python {sys.argv[0]} <model.safetensors>")
        sys.exit(1)

    model_path = sys.argv[1]

    with safe_open(model_path, framework="pt") as f:
        print(f"Model: {model_path}")
        print(f"Number of tensors: {len(list(f.keys()))}")
        print()

        for key in sorted(f.keys()):
            shape = tuple(f.get_tensor(key).shape)
            print(f"{str(shape):20} {key}")

if __name__ == "__main__":
    main()

python layers.py anima-base-v1.0.safetensors

>ctq -h
usage: ctq [-h] -i INPUT [-o OUTPUT] [--comfy_quant] [--simple] [--full_precision_matrix_mult] [--heur] [--device DEVICE]
           [--verbose {DEBUG,VERBOSE,NORMAL,MINIMAL}] [--manual_seed MANUAL_SEED] [--use_speed] [--save-quant-metadata] [--verbose-pinned] [--low-memory]

Convert safetensors weights to Scaled FP8 format.

Default behavior: FP8 quantization with per-tensor scaling.
For INT8 and other experimental options, see --help-experimental.
For model-specific layer exclusions, see --help-filters.
For advanced LR tuning and early stopping, see --help-advanced.
For conversion and utility modes, see --help-modes.

Standard Options:
  -h, --help            show this help message and exit
  -i INPUT, --input INPUT
                        Input safetensors file path.
  -o OUTPUT, --output OUTPUT
                        Output safetensors file path. Auto-generated if not provided.
  --comfy_quant, --comfy-quant
                        Use Comfy quantization method.
  --simple              Skip SVD optimization, use simple quantization.
  --full_precision_matrix_mult, --full-precision-matrix-mult, -fpmm
                        Add full_precision_matrix_mult=True to .comfy_quant metadata.
  --heur                Skip layers with poor quantization characteristics (aspect ratio, size).
  --device DEVICE       Device to use for quantization (e.g., 'cpu', 'cuda'). Overrides auto-detection. Recommended with --simple for FP8/INT8.
  --verbose {DEBUG,VERBOSE,NORMAL,MINIMAL}
                        Set verbosity: NORMAL (default), VERBOSE (increased), MINIMAL (reduced), DEBUG (all).
  --manual_seed MANUAL_SEED, --manual-seed MANUAL_SEED
                        Set a manual seed for reproducibility. Use -1 for random.
  --use_speed, --use-speed
                        [Prodigy] Enabled the use_speed parameter for Prodigy optimizer.
  --save-quant-metadata, --save_quant_metadata
                        Save quantization metadata in safetensors header (under _quantization_metadata key)
  --verbose-pinned, --verbose_pinned
                        Print per-tensor pinned memory transfer details
  --low-memory, --low_memory, -lm
                        Use streaming tensor loading to reduce RAM usage (recommended for models >50% of available RAM)

>ctq -hf
Model-Specific Exclusion Filters
============================================================

These flags keep certain model-specific layers in high precision
(not quantized). Multiple filters can be combined.

Text Encoders:
----------------------------------------
  --gemma4                       Gemma4 text/multimodal model: skip audio, per_layer_input_gate, per_layer_projection, vision, multi_modal_projector
  --qwen35                       Qwen2.5 text/multimodal model: skip first/last layers, embeddings, visual components
  --t5xxl                        T5-XXL text encoder: skip norms/biases, remove decoder layers
  --mistral                      Mistral text encoder exclusions
  --visual                       Visual encoder: skip MLP layers (down/up/gate proj)
  --generic_text                 Generic text encoder: skip MLP layers (down/up/gate proj)

Diffusion Models (Flux-style):
----------------------------------------
  --anima                        Anima diffusion model: keep first blocks, adaln_modulation, final/embedding layers high-precision
  --lens                         LENS diffusion model: keep time_text_embed, img_in, norm_out, proj_out, some mod layers high-precision
  --flux2                        Flux.2: keep modulation/guidance/time/final layers high-precision
  --distillation_large           Chroma/distilled (large): keep distilled_guidance, final, img/txt_in high-precision
  --distillation_small           Chroma/distilled (small): keep only distilled_guidance high-precision
  --nerf_large                   NeRF (large): keep nerf_blocks, distilled_guidance, txt_in high-precision
  --nerf_small                   NeRF (small): keep nerf_blocks, distilled_guidance high-precision
  --radiance                     Radiance model: keep img_in_patch, nerf_final_layer high-precision
  --krea2                        Krea2: keep firs, las, tml, txtfusion, last.modulation, tpro layers high-precision
  --ideogram4                    Ideogram4: keep embed_image_indicator, t_embedding, adaln_proj, final_layer, input_proj layers high-precision

Video Models:
----------------------------------------
  --wan                          WAN video model: skip embeddings, encoders, head
  --hunyuan                      Hunyuan Video 1.5: skip layernorm, attn norms, vision_in
  --ltxv2                        LTXv2: keep some transformer blocks high-precision and exclude vae and vocoder

Image Models:
----------------------------------------
  --qwen                         Qwen Image: skip added norms, keep time_text_embed high-precision
  --zimage                       Z-Image: skip cap_embedder/norms, keep x_embedder/final high-precision
  --zimage_refiner               Z-Image Refiner: keep context/noise refiner high-precision
  --boogu                        Boogu: keep image_index_embedding, ref_image_patch_embedder, time_caption_embed, x_embedder high-precision

ANIMA_LAYER_KEYNAMES = [
    "net.blocks.0.", "net.blocks.1.adaln_modulation", "final_layer", "llm_adapter", "t_embedder", "x_embedder"
]

cd ComfyUI
.\venv\Scripts\activate.bat
git clone https://github.com/Comfy-Org/comfy-model-tools

>python quant_int8_convrot.py -h
usage: quant_int8_convrot.py [-h] [--dry-run] [--exclude EXCLUDE] [--include INCLUDE] [--min-gemm MIN_GEMM]
                             [--mseclip] [--downcast-fp32] [--warn-thresh WARN_THRESH] [--verify-report VERIFY_REPORT]
                             src [dst]

positional arguments:
  src
  dst                   output .safetensors; if omitted, derived from SRC by replacing bf16/fp16/fp32 with
                        int8_convrot (or appending _int8_convrot)

options:
  -h, --help            show this help message and exit
  --dry-run             report the plan, write nothing
  --exclude EXCLUDE     regex; matching layers are FORCED to passthrough
  --include INCLUDE     regex; matching eligible layers are FORCED to quantize
  --min-gemm MIN_GEMM   skip a layer if min(N,K) < this (default 256: a GEMM whose small side is under ~256 never
                        beats bf16 at any M, so int8 is pure overhead). --min-gemm 0 disables.
  --mseclip             MSE-optimal clip instead of absmax (~2-3% lower weight error, but a proxy — validate output
                        before trusting it)
  --downcast-fp32       downcast stray fp32 passthrough linears to compute dtype
  --warn-thresh WARN_THRESH
                        warn on any quantized layer whose relerr% exceeds this (default 2.0)
  --verify-report VERIFY_REPORT
                        write the full per-lay

EXCLUDE_SEG = re.compile(
    r"scale_shift|rope|rotary|rel_pos|pos_?embed|embedder|"
    r"gate_logits|router|routing|logit|temperature|"
    r"(?:^|_)time|temb|t_emb|guidance|register|refiner_blocks|adapter|"
    r"(?:^|_)(?:final|head|proj_out|out_layer)(?:_|$)")

        elif model_type == "anima":
            Int8TensorwiseOps.excluded_names = [
                'embed', 'llm', 'adaln',
            ]

EXCL: Final[tuple[str]] = (
    "embed",
    "bias",
    "norm",
    "scale",
    "llm",
    "adaln",
    "first_stage_model",
    "cond_stage_model",
    "vae",
    "text",
    "time",
)

    "Anima": {
      "_note": "Anima (Cosmos-based) requires the 'llm_adapter' keys, biases, and normalizations to remain unquantized to prevent color banding and generation artifacts.",
      "blacklist": ["bias", "norm", "scale", "llm_adapter", "final_layer", "proj_out", "x_embedder", "t_embedder", "context_embedder"],
      "fp8_layers": [],
      "preserve_extended_metadata": false
    },

ctq -i anima-base-v1.0.safetensors -o anima-base-v1.0_int8.safetensors --int8 --scaling_mode row --simple --convrot --convrot-group-size 64 --comfy_quant --save-quant-metadata --anima

ctq -i anima_aestheticV10.safetensors -o anima_aestheticV10_int8.safetensors --int8 --scaling_mode row --simple --convrot --convrot-group-size 64 --comfy_quant --save-quant-metadata --anima --exclude-layers "blocks\.0\.|blocks\.1\.adaln_modulation"

ctq -i anima-base-v1.0.safetensors -o anima-base-v1.0_int8.safetensors --int8 --scaling_mode row --convrot --convrot-group-size 64 --comfy_quant --save-quant-metadata --anima

ctq -i anima-base-v1.0.safetensors -o anima-base-v1.0_int8.safetensors --int8 --scaling_mode row --convrot --convrot-group-size 64 --comfy_quant --save-quant-metadata --anima --scale-optimization dualround

python quant_int8_convrot.py anima-base-v1.0.safetensors anima-base-v1.0_int8.safetensors --mseclip

python quant_int8_convrot.py anima-base-v1.0.safetensors anima-base-v1.0_int8.safetensors --mseclip --exclude "blocks\.0\.|blocks\.1\.adaln_modulation"