.GGUF File Format – Loader: key types and tensors

How a key is read

• Exact type. A scalar key is read into a C++ variable; the loader (GGUFMeta::GKV<T>::get_kv) requires the GGUF value type that matches the variable and otherwise stops with key … has wrong type … but expected type …. There is no widening: the matching pairs are bool = BOOL, uint8_t = UINT8, uint16_t = UINT16, uint32_t = UINT32, uint64_t = UINT64, int8_t = INT8, int16_t = INT16, int32_t = INT32, int64_t = INT64, float = FLOAT32, double = FLOAT64, string = STRING. The model code instantiates scalar reads only for bool, float, uint32_t, int32_t and string, so every model key is one of those five GGUF types, except the pooling type (read as UINT32 and converted to an enum, UNSPECIFIED if absent).
• Required or optional. get_key(key, variable, required) throws key not found in model when a required key is missing; with required false it returns false and leaves the variable. Calls with no third argument are required.
• Overrides. A caller can pass key overrides (llama_model_kv_override): a value of type bool, integer, float or string replaces the file's value for the matching variable type; one of the wrong override type is ignored with a warning.
• Arrays (get_arr): the key must be an ARRAY; the element type must be UINT32 or INT32 (read into int32_t or uint32_t), UINT64 (uint64_t), FLOAT32 (float) or STRING (std::string); fixed-size array targets also accept BOOL elements (stored as 0 or 1 in int32_t or uint32_t). Any other element type is an error. A fixed-size target rejects an array longer than its capacity. get_arr_n reads only the array's length into an unsigned integer.
• Scalar or array (get_key_or_arr): for per-layer values the key may be a single value (repeated for every layer) or an array whose length must equal the number of layers, else wrong array length.
• Split files. split.count is read as UINT16 (optional; absent means one file). When it is above 1, split.no (UINT16, required) must be 0 in the file that is opened, or the loader stops with illegal split file idx. split.tensors.count (INT32) is required when a split file was skipped, and when there are several splits and no extra metadata files; otherwise it is optional and, if present, is compared with the number of tensors found (corrupted model: … tensors expected). These keys are not model parameters.
• Tensors. A tensor's file name is the template of LLM_TENSOR_NAMES filled with the layer number, plus .weight or .bias (for example blk.0.attn_q.weight). A tensor created with the flag TENSOR_NOT_REQUIRED may be absent; TENSOR_DUPLICATED reuses another tensor's data (for example the output matrix from the token embedding when the file has no output.weight). The expected dimensions are fixed by the architecture's code.

Keys every architecture reads

llama_model_base::load_hparams reads these first (all are stored in hparams unless noted). Keys under {arch}. use the architecture name. "Optional" means the call passes required = false.

Value type of the other keys, by C++ destination

Each architecture's load_arch_hparams reads further keys into fields of hparams. The GGUF type follows the field's type in llama-hparams.h. The lists name the key without its {arch}. prefix unless it has none. Whether a call is required is given per architecture in Required keys of each architecture; every call not listed there passes required = false. Keys whose destination is a local variable are listed last, with the variable's type.

• UINT32 (uint32_t): vocab_size (also read from the array length of tokenizer.ggml.tokens), leading_dense_block_count, attn_res.block_size, feed_forward_length forms above, expert_feed_forward_length (array form allowed), expert_shared_feed_forward_length, expert_chunk_feed_forward_length, expert_shared_count, expert_latent_length, experts_per_group, expert_gating_func (1 softmax, 2 sigmoid, 4 square-root softplus per the writer's constants), moe_every_n_layers, moe_latent_size, interleave_moe_layer_step, n_deepstack_layers, embedding_length_per_layer_input, rescale_every_n_layers, time_mix_extra_dim, time_decay_extra_dim, wkv.head_size, token_shift_count, decoder_block_count, dense_2_feat_in, dense_2_feat_out, dense_3_feat_in, dense_3_feat_out, shortconv.l_cache, realtime_delay_tokens, attention.sliding_window, attention.relative_buckets_count, attention.group_norm_groups, attention.q_lora_rank, attention.kv_lora_rank, attention.kv_lora_rank_swa, attention.key_length_mla, attention.value_length_mla, attention.key_length_mla_swa, attention.value_length_mla_swa, attention.decay_lora_rank, attention.iclr_lora_rank, attention.value_residual_mix_lora_rank, attention.gate_lora_rank, attention.pass_lora_rank, attention.value_expert_count, attention.value_expert_used_count, attention.temperature_length, attention.output_group_count, attention.output_lora_rank, attention.indexer.head_count, .key_length, .top_k, .block_size, .local_blocks, .kpool, .candidate_top_k_blocks, .candidate_block_size, ssm.conv_kernel, ssm.inner_size, ssm.state_size, ssm.time_step_rank, ssm.group_count, kda.head_dim, hrm.layers_per_stack, hrm.h_cycles, hrm.l_cycles, metis.pass_count, metis.memory_length, metis.route_length, mudd.feed_forward_length, mudd.tap_count, hyper_connection.count, hyper_connection.sinkhorn_iterations, hyper_connection.low_rank, hash_layer_count, engram.pad_token_id, ple.ngram_size, ple.heads_per_ngram, ple.conv_kernel, ple.eos_token_id, ple.image_token_id, the dflash keys block_size, conv_kernel_size, conv_group_size, selector_rank, selector_top_k, lilicorr.hidden_size, lilicorr.candidate_topk, lilicorr.num_heads, lilicorr.num_layers, decision.block_count, decision.routing_block_count, decision.head_count.
• INT32 (int32_t): attention.indexer.candidate_source_layer (default −1).
• FLOAT32 (float): attention.layer_norm_epsilon, attention.layer_norm_rms_epsilon, attention.group_norm_epsilon, attention.max_alibi_bias, attention.clamp_kqv, attention.scale, attention.output_scale, attention.value_scale, attention.temperature_scale, attention.xsa_epsilon, attention.compress_rope_freq_base, attn_logit_softcapping (default 50), router_logit_softcapping (30), final_logit_softcapping (30), post_norm_epsilon, logit_scale, residual_scale, embedding_scale, expert_group_scale (0.05), expert_weights_scale, ssm.norm_epsilon, kda.gate_lower_bound, activation.situ_beta, activation.situ_linear_beta, metis.continuation_logit_limit, hyper_connection.epsilon, hyper_connection.magnitude, rope.freq_base_swa (10000), rope.scaling.yarn_log_multiplier, rope.scaling.yarn_ext_factor (−1), rope.scaling.yarn_attn_factor (1), rope.scaling.yarn_beta_fast (32), rope.scaling.yarn_beta_slow (1), lilicorr.vector_eps, lilicorr.logit_scale.
• BOOL (bool): use_parallel_residual, swin_norm, norm_before_residual, norm_before_fc, expert_weights_norm, attention.indexer.kpool_select_tail (default true), ssm.dt_b_c_rms, kda.safe_gate, hrm.prefix_lm, hyper_connection.single_pass.
• STRING: hidden_activation (read into a string field, destination not checked), tokenizer.chat_template, tokenizer.ggml.model, tokenizer.ggml.pre, tokenizer.huggingface.json, tokenizer.ggml.metaspace.prepend_scheme.
• Arrays read with get_arr or get_key_or_arr: attention.sliding_window_pattern (per-layer flags, UINT32/INT32/BOOL elements), attention.rope_pattern, attention.recurrent_layers, attention.indexer.types, attention.indexer.source_layers, attention.kv_source_map, attention.kv_source_layers, attention.compress_ratios, rope.dimension_sections, deepstack_mapping, final_logit_sigmoid_capping (FLOAT32), swiglu_clamp, swiglu_clamp_exp, swiglu_clamp_shexp (scalar or array, FLOAT32), xielu.alpha_n, xielu.alpha_p, xielu.beta, xielu.eps (scalar or array of FLOAT32; these four have no architecture prefix), ple.layers, ple.layer_multipliers, ple.head_offsets, ple.head_vocab_sizes (UINT64 for the head arrays), mudd.tap_indices, target_layers, adapters.token_ids_activate, adapters.token_ids_substitute, general.tensor_extra.name (STRING), engram.token_map, prism.hadamard.sign_widths, .sign_values, .weight_names, .inverse_weight_names (STRING).
• Keys with a local or nested destination, settled from the code that reads them. UINT32: general.file_type (optional; read by the loader into the file-type enumeration, overriding the type the loader guessed from the tensors), full_attention_interval (local variable, read only when attention.recurrent_layers is absent, and used to mark every layer except each Nth as recurrent; required in clover-moe, optional in the others, with default 8 in minimax-01; zero is rejected in clover-moe and qwen4exp), num_loops (optional, default 1, at least 1; the layer count times the loop count must not exceed LLAMA_MAX_LAYERS), attention.shared_kv_layers (optional, default 0), decoder_start_token_id (optional), target_hidden_size (eagle3, required), lilicorr.factor_dim and lilicorr.mlp_hidden_size (dflash, required when the model has LiLiCorr layers; zero is an error), adapters.count and adapters.lora_rank (granite-switch, required; each at most 4096), and posnet.embedding_length, posnet.block_count, convnext.embedding_length, convnext.block_count (fields of hparams.posnet and hparams.convnext, which are uint32_t; read only for WavTokenizer, as in the table above). BOOL: skip_loop_final_norm (optional, default false), prism.hadamard.tied_output and prism.hadamard.gdn_v_grouped (optional, default false). FLOAT32: rope.freq_base_spatial (sensenova-u1, required; default of the field 10000 is overwritten), adapters.router_gain (granite-switch, optional, default 15). The Prism Hadamard keys are read for every architecture, before the base keys (see below).

prism.hadamard.* (no architecture prefix). Only prism.hadamard.version (UINT32, optional) switches the group on; without it the other keys are not read (except tied_output, which is read first). Version must be 1 or 2, and version 2 requires tied_output true (and then the file must have no output.weight), version 1 requires it false. With a version present the following are required: block_size (UINT32, a power of two), transform (STRING, must be normalized-sylvester-walsh-hadamard), axis (STRING, must be input-last-dimension), sign_mode (STRING, identity or explicit), weight_names (array of STRING, not empty). With sign_mode explicit, sign_widths and sign_values (arrays whose elements are INT32 or UINT32, each value +1 or −1, the widths multiples of the block size and summing to the length of the values) are required. inverse_weight_names (array of STRING, optional) may name only token_embd.weight. The group is accepted only for the architectures llama, qwen3, qwen3moe, qwen35, qwen35moe and qwen3next; any other fails with arch … is not verified to apply the activation transform.

General, sampling, adapter and tokenizer keys

These keys have no architecture prefix (except the {arch}.adapters.* ones above). They are read in src/llama-vocab.cpp, src/llama-model.cpp, src/llama-model-loader.cpp, src/llama-adapter.cpp, src/llama-impl.cpp and, for the sampling defaults, common/common.cpp (read from the same branch on 2026-11-13).

general.*

• general.architecture (STRING): read with the optional form into the architecture name, which then selects the key prefix and the file in src/models; an unknown name stops the load with unknown model architecture.
• general.name (STRING, optional): the model name; also used by the tokenizer loader, in lower case, to choose some pre-tokenizer workarounds.
• general.file_type (UINT32, optional): see above.
• general.type (STRING): not read for a model; for a LoRA adapter it must be adapter.
• general.tensor_extra.name (array of STRING, optional) and general.tensor_extra.prec_a4 (array of BOOL, required when the first is present, with the same length): the names of tensors and, for each, whether it may use 4-bit activations; the tensors with false keep a higher precision for their second operand.
• general.alignment, general.quantization_version, general.author, general.version, general.url, general.description, general.license, general.source.url, general.source.huggingface.repository: no call in the files read. Every non-array key of the file, whatever its name, is also kept as text (below).

Every key as text

While loading the hyperparameters, the model keeps a map from key name to a string for every key whose value is not an array (gguf_kv_to_str): a STRING as it is, integers in decimal, FLOAT32 and FLOAT64 with std::to_string (six digits after the point), BOOL as true or false. Arrays are left out of the model's map (the log shows them in brackets). The functions llama_model_meta_val_str and llama_model_chat_template read from this map, so these keys may have any non-array type.

general.sampling.*

The library itself never reads them. The helper common_init_sampler_from_model in common/common.cpp, used by llama.cpp's command-line programs, looks each key up through llama_model_meta_val_str (so the GGUF type may be any non-array type, because the lookup goes through the text above), and applies it only if the user did not give the matching command-line option. A missing key leaves the program's own default. Numbers are parsed from the text with strtol (base 10) or strtof; a value that does not start with a number is ignored.

adapter.*

A LoRA adapter file is a GGUF file loaded by llama_adapter_lora_init (src/llama-adapter.cpp). It reads its metadata with the plain GGUF calls, not the model loader's typed calls: a key that is absent gives an empty string or 0 (never an error by itself), but the wrong type for the call stops the load.

The adapter's tensors are named <base tensor name>.lora_a and .lora_b; a name ending in _norm.weight is skipped, and any other name is an error. Every non-array key is also kept as text. The {arch}.adapters.* keys of the model file (granite-switch) are a different thing: they describe adapters built into a model file.

tokenizer.*

llama_vocab::impl::load reads the vocabulary in this order. "Required" counts only for a model file with a vocabulary; with tokenizer.ggml.model equal to no_vocab or none the loader reads only tokenizer.ggml.pre, tokenizer.ggml.token_type_count and the optional UINT32 {arch}.vocab_size (it then creates that many empty tokens) and returns.

Required keys of each architecture

Every get_key, get_key_or_arr and get_arr call in load_arch_hparams of each file of src/models/ was compared with its source on raw.githubusercontent.com. A call is required when it has no third (or last) argument; a call whose last argument is false (or 0) is optional and is not listed. A required call that sits in an if is needed only on that path (for example clover-moe needs full_attention_interval only when attention.recurrent_layers is absent). The base keys of the table above are required or optional as shown there for every architecture; this section lists what each architecture adds. Key names are without the {arch}. prefix, except those marked "no prefix".

Only the RMS epsilon (attention.layer_norm_rms_epsilon) is required: arcee, arctic, baichuan, bitnet, chameleon, chatglm, cogvlm, deci, dream, eurobert, exaone, exaone4, gemma, gemma2, gemma3, glm4, grok, hunyuan-vl, internlm2, llada, llada-moe, llama, maincoder, minicpm, mistral3, nanbeige, neo-bert, olmo2, olmoe, openelm, pangu-embed, phi3, phimoe, plamo, plamo3, qwen, qwen2, qwen2moe, qwen3, qwen3moe, qwen3vl, qwen3vlmoe, refact, rnd1, seed-oss, smallthinker, smollm3, xverse.

Only the layer-norm epsilon (attention.layer_norm_epsilon) is required: bert, bloom, codeshell, command-r, falcon, gpt2, jais, jais2, jina-bert-v2, jina-bert-v3, modern-bert, mpt, nemotron, nomic-bert, nomic-bert-moe, olmo, orion, phi2, pockettts, stablelm, starcoder, starcoder2, vllm-decision2.

Architectures that need more. "RMS" or "LN" says which epsilon is also required (the same two keys).

A file of src/models/ that is in neither list and not in the table has no required call of its own (for instance llama-embed, which reuses the LLaMA code, or clip). The calls that are optional in every file (for example leading_dense_block_count in most) are not repeated here. The mechanical way to check this list is the pattern ml.get_(key|key_or_arr|arr|arr_n)(LLM_KV_… without a final false or 0 in each file; deepseek32, deepseek4 and dflash pass 0 or 0.0f in places, which converts to required = false.

Tensor name templates

LLM_TENSOR_NAMES maps each tensor identifier to a template; %d is the layer (or, in posnet, convnext and the ffn_*.%d per-expert forms, the block or expert number). Identifiers below are written without the LLM_TENSOR_ prefix and given as IDENTIFIER = template; templates starting blk.N. are per layer.

Global: TOKEN_EMBD = token_embd; OUTPUT_NORM = output_norm; OUTPUT_NORM_LFM2 = token_embd_norm (a workaround for a wrongly named tensor); OUTPUT = output; ROPE_FREQS = rope_freqs; ROPE_FACTORS_LONG = rope_factors_long; ROPE_FACTORS_SHORT = rope_factors_short; POS_EMBD = position_embd; TOKEN_EMBD_NORM = token_embd_norm; TOKEN_TYPES = token_types; CLS = cls; CLS_OUT = cls.output; CLS_NORM = cls.norm; HRM_Z_L_INIT = hrm.z_l_init; STREAM_EMBD = stream_embd; VALUE_EMBD = value_embd; ENC_OUTPUT_NORM = enc.output_norm; DEC_OUTPUT_NORM = dec.output_norm; DENSE_2_OUT = dense_2; DENSE_3_OUT = dense_3; OUTPUT_RES_SCORE = output_res_score; CONV1D = conv1d; FC = fc; D2T = d2t; MASKED_EMBD_CENTROIDS = masked_embd_centroids; MASKED_EMBD_ORDERING = masked_embd_ordering; PER_LAYER_TOKEN_EMBD = per_layer_token_embd; PER_LAYER_MODEL_PROJ = per_layer_model_proj; PER_LAYER_PROJ_NORM = per_layer_proj_norm; ALTUP_UNEMBD_PROJ = altup_unembd_proj; ALTUP_PROJ = altup_proj; NEXTN_PROJ_PRE = nextn.pre_projection; NEXTN_PROJ_POST = nextn.post_projection; DSPARK_MARKOV_W1 = markov_w1; DSPARK_MARKOV_W2 = markov_w2; DSPARK_CONF_PROJ = conf_proj; DSPARK_JOINT_PROJ = joint_proj; DSPARK_MASK_EMBD = mask_embd; DFLASH_SELECTOR_PREV = selector_predecessor; DFLASH_SELECTOR_NEXT = selector_successor; DFLASH_SELECTOR_HIDDEN = selector_hidden; HC_HEAD_FN = output_hc_fn; HC_HEAD_BASE = output_hc_base; HC_HEAD_SCALE = output_hc_scale; HC_HEAD_NORM = output_hc_norm; HC_HEAD_DOWN = output_hc_down; HC_HEAD_UP = output_hc_up; CONT_HIDDEN_STATE, CONT_HIDDEN_RETR, CONT_HIDDEN_DIFF, CONT_HIDDEN_ROUTE, CONT_HIDDEN, CONT_OUTPUT = the same names in lower case (cont_hidden_state …); DEPTH_MEM_STATE_WRITE, _META_WRITE, _ENTRY_BIAS, _QUERY, _KEY, _VALUE, _OUT, _ROUTE_STATE, _ROUTE_RETR, _ROUTE_DIFF, _ROUTE_HIST, _ROUTE, _STREAM_GATE = depth_mem_ plus the lower-case suffix (depth_mem_state_write …); DECISION_* = decision.hidden_norm, proj_memory, proj_question, proj_option_question, proj_global, proj_option_context, proj_option_lexical, option_summary_norm, field_norm, option_norm, scales, scorer, scorer_out, query_norm, key, query, option_mlp, query_mlp; LILICORR_* = lilicorr.token_proj, pass_hidden_proj, slot_embedding, rank_embedding, attention_bias, context_proj, output_norm, anchor_norm, factor_input_proj, out_head, in_head, anchor_out_head, and LILICORR_ATTN_IN_PROJ = lilicorr.layers.%d.attn.in_proj.

Attention (blk.N.): ATTN_NORM = attn_norm; ATTN_NORM_2 = attn_norm_2; ATTN_Q = attn_q; ATTN_K = attn_k; ATTN_V = attn_v; ATTN_QKV = attn_qkv; ATTN_OUT = attn_output; ATTN_ROT_EMBD = attn_rot_embd; ATTN_POST_NORM = post_attention_norm; ATTN_Q_NORM = attn_q_norm; ATTN_K_NORM = attn_k_norm; ATTN_OUT_NORM = attn_output_norm; ATTN_GATE = attn_gate; ATTN_RESID_ALPHA / _BETA = attn_resid_alpha / attn_resid_beta; ATTN_VRES_LAMBDA = attn_vres_lambda; ATTN_Q_A_NORM = attn_q_a_norm; ATTN_KV_A_NORM and ATTN_KV_NORM = attn_kv_a_norm; ATTN_Q_A = attn_q_a; ATTN_Q_B = attn_q_b; ATTN_KV_A_MQA = attn_kv_a_mqa; ATTN_KV_B = attn_kv_b; ATTN_KV = attn_kv; ATTN_K_B = attn_k_b; ATTN_V_B = attn_v_b; ATTN_OUT_A / _B = attn_output_a / attn_output_b; ATTN_SUB_NORM = attn_sub_norm; ATTN_SINKS = attn_sinks; ATTN_SSMAX = attn_ssmax; ATTN_QKV_GATE = attn_qkv_gate; ATTN_V_GATE = attn_v_gate; ATTN_V_EXPS = attn_v_exps; ATTN_VE_GATE = attn_ve_gate; ATTN_XSA_ALPHA = attn_xsa_alpha; ATTN_RESID_SCALE = attn_resid_scale; ATTN_RES_SCORE = attn_res_score; ATTN_PASS_LORA_DOWN / _UP = attn_pass_lora_down / attn_pass_lora_up; ATTN_PASS_GATE = attn_pass_gate; ATTN_COMPRESSOR_WKV = attn_compressor_kv; ATTN_COMPRESSOR_WGATE = attn_compressor_gate; ATTN_COMPRESSOR_APE = attn_compressor_ape; ATTN_COMPRESSOR_NORM = attn_compressor_norm; LAYER_OUT_NORM = layer_output_norm; LAYER_OUT_SCALE = layer_output_scale; INDEXER_K_NORM = indexer.k_norm; INDEXER_PROJ = indexer.proj; INDEXER_ATTN_K = indexer.attn_k; INDEXER_ATTN_Q_B = indexer.attn_q_b; INDEXER_Q_PROJ = indexer.q_proj; INDEXER_K_PROJ = indexer.k_proj; INDEXER_Q_NORM = indexer.q_norm; INDEXER_SSMAX = indexer.ssmax; INDEXER_COMPRESSOR_WKV = indexer_compressor_kv; INDEXER_COMPRESSOR_WGATE and INDEXER_KPOOL_GATE = indexer_compressor_gate; INDEXER_COMPRESSOR_APE and INDEXER_KPOOL_APE = indexer_compressor_ape; INDEXER_COMPRESSOR_NORM = indexer_compressor_norm.

Feed-forward (blk.N.): FFN_NORM = ffn_norm; FFN_GATE = ffn_gate; FFN_DOWN = ffn_down; FFN_UP = ffn_up; FFN_ACT = ffn.act; FFN_GATE_INP = ffn_gate_inp; FFN_GATE_INP_SHEXP = ffn_gate_inp_shexp; FFN_GATE_EXP / FFN_DOWN_EXP / FFN_UP_EXP = ffn_gate.E / ffn_down.E / ffn_up.E (one tensor per expert E); FFN_GATE_EXPS = ffn_gate_exps; FFN_GATE_UP_EXPS = ffn_gate_up_exps; FFN_DOWN_EXPS = ffn_down_exps; FFN_UP_EXPS = ffn_up_exps (one tensor holding all experts); FFN_GATE_SHEXP / FFN_UP_SHEXP / FFN_DOWN_SHEXP = ffn_gate_shexp / ffn_up_shexp / ffn_down_shexp (shared expert); FFN_GATE_CHEXPS / FFN_DOWN_CHEXPS / FFN_UP_CHEXPS = ffn_gate_chexps and the like; FFN_EXP_PROBS_B = exp_probs_b; FFN_EXP_PROBS_B_VL = exp_probs_b_vl; FFN_GATE_TID2EID = ffn_gate_tid2eid; FFN_LATENT_DOWN / FFN_LATENT_UP = ffn_latent_down / ffn_latent_up; FFN_ROUTED_DOWN / _UP / _NORM = ffn_routed_down / ffn_routed_up / ffn_routed_norm; FFN_EXP_EMBD = ffn_exp_embd; FFN_PRE_EXP_NORM = ffn_pre_exp_norm; FFN_POST_EXP_NORM = ffn_post_exp_norm; FFN_NORM_EXPS = ffn_norm_exps; FFN_POST_NORM = post_ffw_norm; FFN_POST_NORM_1 / _2 = post_ffw_norm_1 / post_ffw_norm_2; FFN_PRE_NORM_2 = pre_ffw_norm_2; FFN_NORM_ADA_DOWN / _UP = ffn_norm_ada_down / ffn_norm_ada_up; FFN_SUB_NORM = ffn_sub_norm; FFN_RESID_ALPHA / _BETA / _SCALE = ffn_resid_alpha / ffn_resid_beta / ffn_resid_scale; FFN_RES_SCORE = ffn_res_score; VISEXP_ATTN_QKV = vis_attn_qkv; VISEXP_ATTN_OUT = vis_attn_output; VISEXP_FFN_GATE / _DOWN / _UP = vis_gate / vis_down / vis_up; MUDD_DOWN / MUDD_UP = mudd_down / mudd_up; ENGRAM_KV / ENGRAM_GATE = engram_kv / engram_gate; DFLASH_ATTN_CONV_BASE / _PROJ = attn_conv_base / attn_conv_proj; DFLASH_FFN_CONV_BASE / _PROJ = ffn_conv_base / ffn_conv_proj.

Hyper-connections (blk.N.): HC_ATTN_MIX, HC_ATTN_READ, HC_ATTN_WRITE, HC_FFN_MIX, HC_FFN_READ, HC_FFN_WRITE, HC_ATTN_FN, HC_ATTN_BASE, HC_ATTN_SCALE, HC_FFN_FN, HC_FFN_BASE, HC_FFN_SCALE, HC_ATTN_NORM, HC_ATTN_DOWN, HC_ATTN_UP, HC_ATTN_INJECT, HC_FFN_NORM, HC_FFN_DOWN, HC_FFN_UP, HC_FFN_INJECT, HC_ATTN_DOWN_INJECT, HC_FFN_DOWN_INJECT = hc_attn_mix, hc_attn_read, hc_attn_write, hc_ffn_mix, hc_ffn_read, hc_ffn_write, hc_attn_fn, hc_attn_base, hc_attn_scale, hc_ffn_fn, hc_ffn_base, hc_ffn_scale, hc_attn_norm, hc_attn_down, hc_attn_up, hc_attn_inject, hc_ffn_norm, hc_ffn_down, hc_ffn_up, hc_ffn_inject, hc_attn_down_inject, hc_ffn_down_inject.

State-space, recurrent and short-convolution (blk.N.): SSM_A_NOSCAN and SSM_A = ssm_a; SSM_CONV1D = ssm_conv1d; SSM_DT = ssm_dt; SSM_BETA_ALPHA = ssm_ba; SSM_ALPHA = ssm_alpha; SSM_IN = ssm_in; SSM_NORM = ssm_norm; SSM_OUT = ssm_out; SSM_X = ssm_x; SSM_D = ssm_d; SSM_DT_NORM / SSM_B_NORM / SSM_C_NORM = ssm_dt_norm / ssm_b_norm / ssm_c_norm; SSM_CONV1D_Q / _K / _V = ssm_conv1d_q / ssm_conv1d_k / ssm_conv1d_v; SSM_F_A / SSM_F_B = ssm_f_a / ssm_f_b; SSM_BETA = ssm_beta; SSM_G_A / SSM_G_B / SSM_G = ssm_g_a / ssm_g_b / ssm_g; SSM_BETA_ALPHA_CAT = ssm_beta_alpha; SHORTCONV_CONV = shortconv.conv; SHORTCONV_INPROJ = shortconv.in_proj; SHORTCONV_OUTPROJ = shortconv.out_proj. RWKV: TIME_MIX_W1, _W2, _LERP_X, _LERP_W, _LERP_K, _LERP_V, _LERP_R, _LERP_G, _LERP_FUSED, _FIRST, _DECAY, _DECAY_W1, _DECAY_W2, _KEY, _VALUE, _RECEPTANCE, _GATE, _LN, _OUTPUT, _W0, _A0, _A1, _A2, _V0, _V1, _V2, _G1, _G2, _K_K, _K_A, _R_K = time_mix_ plus the lower-case suffix (time_mix_w1, time_mix_lerp_x …, time_mix_ln, time_mix_output); CHANNEL_MIX_LERP_K, _LERP_R, _KEY, _VALUE, _RECEPTANCE = channel_mix_lerp_k, channel_mix_lerp_r, channel_mix_key, channel_mix_value, channel_mix_receptance.

Multimodal and other layer groups: CONVNEXT_DW / _NORM / _PW1 / _PW2 / _GAMMA = convnext.N.dw, norm, pw1, pw2, gamma; POS_NET_CONV1, _CONV2, _NORM, _NORM1, _NORM2, _ATTN_NORM, _ATTN_Q, _ATTN_K, _ATTN_V, _ATTN_OUT = posnet.N.conv1, conv2, norm, norm1, norm2, attn_norm, attn_q, attn_k, attn_v, attn_output; PLE_KEY / PLE_VALUE / PLE_NORM_KEY / PLE_NORM_QUERY / PLE_NORM_CONV / PLE_CONV1D = ple_key / ple_value / ple_norm_key / ple_norm_query / ple_norm_conv / ple_conv1d (blk.N.); PER_LAYER_INP_GATE = inp_gate; PER_LAYER_PROJ = proj; PER_LAYER_POST_NORM = post_norm; ALTUP_CORRECT_COEF / _CORRECT_SCALE / _PREDICT_COEF / _ROUTER / _ROUTER_NORM = altup_correct_coef / altup_correct_scale / altup_predict_coef / altup_router / altup_router_norm; LAUREL_L / _R / _POST_NORM = laurel_l / laurel_r / laurel_post_norm (all blk.N.); NEXTN_EH_PROJ, NEXTN_EMBED_TOKENS, NEXTN_ENORM, NEXTN_HNORM, NEXTN_SHARED_HEAD_HEAD, NEXTN_SHARED_HEAD_NORM, NEXTN_HC_HEAD_NORM, NEXTN_HC_HEAD_DOWN, NEXTN_HC_HEAD_UP = blk.N.nextn.eh_proj, embed_tokens, enorm, hnorm, shared_head_head, shared_head_norm, hc_head_norm, hc_head_down, hc_head_up; DENSE and the encoder-decoder families (T5): ENC_ATTN_NORM / _Q / _K / _V / _OUT (= attn_o) / _REL_B (= attn_rel_b) and ENC_FFN_NORM / _GATE / _DOWN / _UP are enc.blk.N. names; DEC_ATTN_NORM / _Q / _K / _V / _OUT (= attn_o) / _REL_B, DEC_CROSS_ATTN_NORM / _NORM_KV / _Q / _K / _V / _OUT (= cross_attn_o) / _REL_B and DEC_FFN_NORM / _GATE / _DOWN / _UP are dec.blk.N. names.

Tensors each architecture creates

For each file of src/models/ (a class llama_model_<name>, usually named like the architecture with hyphens for underscores) the set of identifiers (from the table above) that its load_arch_tensors and graph code mention. The Q, K and V projections of many architectures are created by a shared helper (create_tensor_qkv) and do not appear as ATTN_Q, ATTN_K, ATTN_V in these lists; the dimensions are in the source. A file with no list reuses another class's loader (for example llama-embed reuses the LLaMA one) or has no tensors of its own (clip, whose tensors are in the multimodal projector's own loader). Every file of src/models/ is listed, from afmoe to xverse; the sets were compared with the files on raw.githubusercontent.com (branch master, 2026-11-13).