.SAFETENSORS File Format – Tensors and config

LoRA adapter

A LoRA adapter has no container of its own: it is usually a safetensors file whose tensor names and shapes follow the conventions of the program that trained it. This page follows the code of the two most common writers, Hugging Face PEFT (tuners/lora/layer.py, tuners/lora/config.py, config.py, utils/constants.py, utils/save_and_load.py) and kohya-ss sd-scripts/networks/lora.py, all read 2026-11-12.

The two matrices

For an adapted linear layer with k inputs and d outputs, PEFT creates lora_A as a linear layer from k to r and lora_B from r to d, so the stored weights are r × k and d × r. The layer’s output gains B(A(x)) × scaling, where scaling is lora_alpha / r, or lora_alpha / √r with use_rslora. kohya-ss calls the same pair lora_down (A) and lora_up (B), uses alpha / lora_dim as the scale, and multiplies by a user multiplier as well. By default both start B at zero, so a fresh adapter changes nothing. kohya-ss gives convolution layers a pair of convolutions instead.

Hugging Face PEFT

FileHolds
adapter_model.safetensorsThe adapter tensors (or adapter_model.bin, a PyTorch file, when not saved as safetensors).
adapter_config.jsonThe LoraConfig as JSON.

Tensor names are the model’s module path with lora_A or lora_B inserted, such as ….q_proj.lora_A.weight. In memory the adapter’s name sits after it (lora_A.default.weight); PEFT removes it when saving. On saving it warns if a lora_A or lora_B tensor is empty or has fewer than two dimensions, a sign of an unsharded save.

adapter_config.json keyMeaning
peft_typeThe method, LORA; the one key PEFT expects every config to have.
task_typeThe kind of task, or none.
base_model_name_or_path, revisionThe base model the adapter was trained on.
rThe rank, default 8.
lora_alphaAlpha, default 8.
lora_dropoutDropout, default 0.
target_modulesThe module names or pattern adapted; all-linear is a shorthand.
biasnone, all or lora_only.
use_rsloraUse lora_alpha / √r as the scale.
modules_to_saveOther modules saved in full with the adapter.
rank_pattern, alpha_patternPer-module overrides of r and lora_alpha.
use_doraWeight-decomposed LoRA (DoRA).

kohya-ss sd-scripts

  • One file holds everything. Each adapted module’s name is a prefix, a dot, the module path, a dot and the child name, with every dot then replaced by an underscore.
  • Prefixes: lora_unet for the image model and lora_te for the text encoder; SDXL uses lora_te1 and lora_te2 for its two text encoders.
  • Each module has .lora_down.weight, .lora_up.weight and .alpha, a single-number tensor.
  • When saving as safetensors, the training metadata goes in the safetensors __metadata__, with sshs_model_hash and sshs_legacy_hash added; other extensions are saved with torch.save.