config.json is a Transformers configuration object written as JSON;
each model family defines its own class and keys. This page follows the base class
in configuration_utils.py,
the file names in utils/constants.py, dynamic_module_utils.py
for auto_map, and
configuration_llama.py
as an example, all read 2026-11-12. Loading a config builds the model’s shape; it does
not load any weights.
Keys from the base class
| Key | Meaning |
|---|---|
model_type | The family identifier, such as llama; written into the JSON from the class and used by AutoConfig to pick the class back. |
transformers_version | The Transformers version that wrote the file. |
architectures | Model classes that can be used with the saved weights. |
dtype | The data type of the weights, written as a name such as float16. Older files call it torch_dtype; Transformers still reads that name but calls it deprecated. |
id2label, label2id, problem_type | Class labels and the kind of classification (regression, single_label_classification or multi_label_classification). |
is_encoder_decoder, output_hidden_states, return_dict, chunk_size_feed_forward | General model behaviour. |
auto_map | For models whose code is not part of Transformers: maps an Auto class to the custom class that implements it. |
The base class lists vocab_size, hidden_size,
num_attention_heads and num_hidden_layers as common
attributes of all subclasses, mapped through each class’s
attribute_map when a model names them differently. It also says that
putting text-generation settings in the model config is deprecated; they belong in
generation_config.json.
Example: LlamaConfig
model_type llama. Its keys and defaults:
| Key | Default |
|---|---|
vocab_size | 32000 |
hidden_size | 4096 |
intermediate_size | 11008 |
num_hidden_layers | 32 |
num_attention_heads | 32 |
num_key_value_heads | none |
head_dim | none |
hidden_act | silu |
max_position_embeddings | 2048 |
initializer_range | 0.02 |
rms_norm_eps | 1e-6 |
use_cache | true |
pad_token_id, bos_token_id, eos_token_id | none, 1, 2 |
pretraining_tp | 1 |
tie_word_embeddings | false |
rope_parameters | none |
attention_bias, mlp_bias | false |
attention_dropout | 0.0 |