# Integrated Explanation of [[Transformer Mechanics]] #paper/generatingphilosophy #llmtext ## Prompt Influence: Soft Programming vs Traditional Code The way a transformer-based language model responds to input is fundamentally shaped by the initial prompt, yet this influence operates more like a 'soft programme' than a set of rigid instructions. This contrasts [[the prompt]]'s effect with traditional computer programmes, which execute based on explicit, discrete logic – conditional statements, loops, function calls – operating on symbolic data according to predefined rules. The transformer model, during inference, does not interpret [[the prompt]] in this symbolic, rule-based manner. Its internal parameters, the weights learned during training, remain fixed. [[The prompt]] does not dynamically rewrite the model's internal code or insert new logical structures. ## Continuous Vector Representation and Attention Instead, [[the prompt]]'s influence is mediated through the continuous, high-dimensional [[vector space]] representations and the differentiable nature of the network's components, particularly the attention mechanism. When the sequence of prompt tokens is fed into the model, each token is converted into an embedding vector, combined with positional information, and then processed through the layers. This initial sequence establishes a specific pattern of activations within the network's neurons. This pattern is not a set of discrete instructions but rather a particular state configuration in the model's high-dimensional representational space. This initial state serves as the starting point for the subsequent computations involved in generating the [[next token]]. The core mechanism enabling [[the prompt]]'s influence is self‑attention. Attention scores ((\alpha_{ij})) are calculated based on the similarity (typically dot products) between Query ((q_i)) and Key ((k_j)) vectors derived from token representations. These scores are continuous values between 0 and 1 after the softmax function. Because these calculations involve continuous vector operations and smooth activation functions (like softmax), small changes in the input representations (originating from [[the prompt]] tokens) can lead to correspondingly smooth, continuous changes in the attention weights assigned throughout the network. This continuous and differentiable nature means [[the prompt]] exerts influence by subtly shifting these weights and activations, rather than by triggering discrete logical branches as in traditional code. There is no symbolic manipulation engine interpreting [[the prompt]]; there is only the flow of activation vectors through learned weights, modulated by attention. Mechanics of self‑attention: For each token representation ((x_i)), the model computes three linear projections—Query ((q_i)), Key ((k_i)) and Value ((v_i)). Relevance between tokens (i) and (j) is measured by the scaled dot‑product (q_i k_j^{\top}/\sqrt{d_k}). The softmax of these scores over all (j) gives attention weights ((\alpha_{ij})). The updated representation for token (i) is the weighted sum (\sum_j \alpha_{ij} v_j). Multi‑head self‑attention repeats this operation in parallel with independent projection matrices, allowing different heads to capture diverse syntactic or semantic relations concurrently. The standard formulation has (O(n^2)) cost in sequence length (n), motivating efficient variants such as sparse or linear attention for long contexts. After the attention sub‑layer, each token is processed by a position‑wise feed‑forward network (FFN) that first projects up to a higher dimension, applies a non‑linearity (commonly GeLU or SwiGLU), then projects back down. FFNs contribute further non‑linear mixing of features and are hypothesised to store a substantial portion of the model's factual knowledge. Both the attention and FFN sub‑layers are wrapped in residual connections ((x + \text{SubLayer}(x))) followed by layer normalisation. These architectural choices stabilise [[the training]] of very deep networks by facilitating gradient flow and keeping activation statistics well‑behaved. Stacking many such layers builds progressively richer representations. ## Attention Landscape Biasing and In-Context Learning The phrase 'biases the high-dimensional attention landscape' refers to how the initial activation pattern set by [[the prompt]] affects subsequent attention calculations. The model's learned weights define a complex landscape of potential computational pathways. [[The prompt]] does not alter this landscape (the weights are fixed), but it determines the starting point and initial trajectory of the computation on this landscape. The activations derived from [[the prompt]] tokens 'prime' or increase the salience of certain learned features or patterns distributed across the network's parameters. When the model calculates attention for subsequent token predictions, the Query vectors generated will interact with the Key vectors presented by all context tokens (including the prompt tokens). The priming effect means that Key vectors associated with the activated pathways are more likely to yield high attention scores when matched with relevant Queries. This effectively biases the attention mechanism to focus on aspects of the context highlighted or made relevant by the prompt's initial activation pattern. This biasing makes certain computational pathways through the network's fixed weights more probable. Pathways associated with the patterns activated by the prompt become more likely to dominate the computation, influencing the final hidden state vector used for prediction. Consequently, the resulting probability distribution over the vocabulary is skewed, making tokens consistent with the activated patterns more likely to be generated. For instance, if a prompt contains examples of a specific task, such as question-answering pairs ('Q: ... A: ... Q: ...'), this sequence establishes a strong activation pattern associated with that format and potentially the underlying task (for example, information retrieval or arithmetic). When the model processes the final question part of the prompt, the attention mechanism is heavily biased by the context of the preceding examples. It is more likely to attend strongly to the relevant parts of the new question and activate the internal processing pathways learned during training for generating answers in the demonstrated format. This is the mechanism behind in-context learning: the examples act as a 'soft programme' by activating the appropriate learned behaviours. Similarly, system instructions or persona prompts (for example, 'You are a helpful assistant', 'Respond in the style of Shakespeare') work by activating the vector patterns and computational pathways that the model learned were statistically associated with helpful responses or Shakespearean language during its training on vast amounts of text data. The initial instruction tokens bias the subsequent generation towards producing output consistent with those activated patterns. ## Mechanics of Next-Token Prediction Following this initial conditioning by the prompt and the processing through the transformer's layers, the core computational task is next-token prediction. After the input sequence (prompt, or prompt plus previously generated tokens) has been processed through the entire stack of transformer blocks, the model needs to produce a probability distribution over possible next tokens. This is typically achieved by taking the final hidden state vector corresponding to the very last token position in the current sequence. This vector is assumed to have accumulated all necessary contextual information to predict the immediate future. This final hidden state vector is then passed through a linear projection layer, parametrised by a weight matrix often denoted (\mathbf{W_o}) (sometimes tied to the input embedding matrix weights). This projection transforms the high-dimensional hidden state into a vector of scores (logits) whose dimensionality equals the size of the model's vocabulary. Finally, a softmax function is applied to these logits. The softmax function exponentiates each logit and normalises the results so that they sum to 1, producing a valid probability distribution over the entire vocabulary. Each element in this distribution represents the model's estimated probability of that specific token being the correct next token in the sequence. ## Model Training and Optimisation The ability to perform this prediction accurately is acquired during the model's training phase. It is optimised using a large corpus of text by minimising a loss function, typically the cross-entropy loss, between the probability distribution it predicts for the next token and the actual next token observed in the training data (represented as a one-hot vector). This method, where the model is always fed the correct preceding sequence to predict the next token, is often called 'teacher forcing'. The cross-entropy loss quantifies the difference between the predicted and actual distributions, heavily penalising the model if it assigns low probability to the true next token. Gradients of this loss function are calculated with respect to all model parameters (embedding weights, QKV projection matrices, FFN weights, and so on) using the back-propagation algorithm. These gradients indicate how each parameter should be adjusted to reduce the prediction error. An optimisation algorithm, such as AdamW, uses these gradients, often averaged over mini-batches of training examples, to iteratively update the model's weights. This process, repeated over trillions of tokens and requiring significant computation, tunes the vast number of parameters throughout the network, refining the attention pathways and processing capabilities to minimise future prediction errors across the diverse patterns present in the training data. Training schedules: Practical optimisation typically employs a learning‑rate schedule with an initial warm‑up phase, a plateau, and a decay (for example, cosine decay). Warm‑up mitigates the risk of unstable updates in the earliest steps, while decay helps the optimiser converge smoothly. Regularisation: Techniques such as dropout, stochastic depth, and weight decay are applied during training to reduce over‑fitting. Dropout randomly zeroes a fraction of activations, encouraging the network to build redundant, robust representations. ## Iterative Text Generation and Decoding Strategies During inference, when the model is used to generate new text, it leverages this learned prediction capability through an iterative loop that proceeds as follows: - The current sequence (initially the prompt, later the prompt plus generated tokens) is fed through the network to compute the probability distribution over the vocabulary for the next token position. - A single token is selected from this distribution using a chosen decoding strategy. Common strategies include greedy decoding (always picking the single most probable token), beam search (keeping track of several high-probability sequences), or sampling methods such as temperature sampling (adjusting the randomness of selection by scaling the logits before softmax) or nucleus (top-p) sampling (sampling only from the smallest set of tokens whose cumulative probability exceeds a threshold). - The selected token is appended to the end of the current sequence. - This extended sequence becomes the input for the next iteration, and the loop continues until a stopping criterion is met (for example, maximum length reached or a special end-of-sequence token generated). The choice of decoding strategy influences the trade-off between output coherence, diversity, and computational cost. ## Reasoning Tokens and Self-Conditioning Within this iterative generation process, specific prompting techniques can induce the model to generate intermediate 'reasoning tokens'. When a user prompts the model using techniques such as 'Let us think step by step' or provides examples demonstrating chain-of-thought reasoning, they are essentially instructing the model to externalise its latent computational process. The intermediate tokens generated by the model in response often reflect the implicit sequence of steps or transformations that the model learned during training were effective for solving complex problems (like arithmetic or multi-hop question answering) present in its training data. It is encouraged to make its internal, learned reasoning process overt in the generated text. Generating these intermediate "reasoning tokens" provides at least two functional benefits. Firstly, it enables self-conditioning. By writing down an intermediate result or step, the model modifies its own context. Subsequent generation steps can then attend to this explicitly stated intermediate information, effectively allowing the model to condition its later computations on its own earlier sub-conclusions. This can help structure the process of solving more complex problems that require maintaining intermediate state. Secondly, it facilitates error-catching and verification. The explicit intermediate steps can be examined (by humans, other automated systems, or potentially the model itself in reflective loops) for logical consistency or factual correctness. This can help identify and potentially correct errors or hallucinations that might occur in a direct, single-step generation, especially when combined with external tools or feedback mechanisms. From a mechanistic standpoint within the transformer architecture, these "reasoning" strings are simply sequences of tokens like any other. They are processed through the same embedding, attention, and FFN layers. Their particular utility arises because they introduce new, explicit content into the sequence that subsequent attention layers can access. By making intermediate computational states persistent and attendable within the sequence context, they effectively augment the model's ability to perform complex, multi-step tasks, analogous to how a human uses an external scratch-pad or shows their working to manage complexity and track progress during problem-solving.