.JSON

Hugging Face model config

Introduction

A weights file such as safetensors holds numbers, not the shape of the network they belong to. In a model folder on Hugging Face, that information is in config.json: which architecture the model uses and how big each part of it is. Transformers reads it first to build an empty model of the right shape, then fills it with the weights. Other programs read it too, for example to convert a model to GGUF.

A model folder often also has generation_config.json, with default settings for generating text, and the tokenizer files described under Hugging Face tokenizer.

Structure

The file is one JSON object. The keys depend on the architecture, but a language model’s config usually includes:

  • architectures: the model class to build, such as LlamaForCausalLM
  • model_type: the family, such as llama, bert or mistral
  • vocab_size, hidden_size, intermediate_size, num_hidden_layers and num_attention_heads: the sizes
  • max_position_embeddings: the longest input it was built for
  • torch_dtype: the number type of the saved weights, such as bfloat16
  • transformers_version: the version that wrote the file

Models whose code is not part of Transformers add an auto_map key naming Python files in the same folder, which run only when the user allows remote code.

Also used for

Links

Software

Reference