.SAFETENSORS File Format

Introduction

Safetensors is a simple format for storing the weights of a neural network, made by Hugging Face in 2022. Earlier PyTorch checkpoints used Python’s pickle format, which can run code when a file is loaded. A safetensors file holds only a description of each tensor and its raw numbers, so loading one is safe, and the data can be mapped straight into memory. It is now the usual format for models shared on Hugging Face, including image generation checkpoints.

The same format also stores fine-tuning adapters, which are much smaller than a model. A LoRA adapter is a safetensors file that holds only the extra low-rank matrices to add to a base model’s layers. Large models are often split into several numbered safetensors files, with a small JSON index that says which file holds each tensor.

Structure

The header maps each tensor’s name to an object with its dtype (such as F32, F16, BF16 or I8), its shape, and data_offsets: the start and end of its bytes, counted from the start of the data. An optional __metadata__ entry holds free text pairs, such as the format of the original checkpoint. The header may be padded with spaces at the end.

Also used for

• LoRA adapter – Small low-rank A and B matrices that adjust a base model’s layers.

Links