You made 2 highlights on the original page that we aren't able to overlay here. View them A note on 'semiotic physics' metasemi 9 mins Feb 9th 2023 Crossposted from the AI Alignment Forum. May contain more technical jargon than usual. Introduction This is an attempt to explain to myself the concept of semiotic physics that appears in the original Simulators post by janus and in a later post by Jan Hendrik Kirchner. Everything here comes from janus and Jan's work, but any inaccuracies or misinterpretations are all mine. TL;DR The prototypical simulator, GPT, is sometimes said to "predict the next token" in a text sequence. This is accurate, but incomplete. It's more illuminating to consider what happens when GPT, or any simulator, is run repeatedly to produce a multi-token forward tra‐ jectory, as in the familiar scenario of generating a text completion in response to a prompt. The token-by-token production of output is stochastic, with a branch point at every step, making the simulator a multiverse gen‐ erator analogous to the time evolution operator of quantum mechanics. In this analogical sense, a simulator such as GPT implements a "physics" whose "elementary particles" are linguistic tokens. When we experience the generated output text as meaningful, the tokens it's composed of are serving as semiotic signs. Thus we can refer to the simulator's physics-analogue as semiotic physics. We can explore the simulator's semiotic physics through experi‐ mentation and careful observation of the outputs it actually pro‐ duces. This naturalistic approach is complementary to analysis of the model's architecture and training. Though GPT's outputs often contain remarkable renditions of the real world, the relationship between semiotic physics and quantum mechanics remains analogical. It's a misconception to think of semiotic physics as a claim that the simulator's semantic world approximates or converges on the real world.[1] Trajectories GPT, the prototypical simulator, is often said to "predict the next token" in a sequence of text. This is true as far as it goes, but it only partially describes typical usage, and it misses a dynamic that's essential to GPT's most impressive performances. Usually, we don't simply have GPT pre‐ dict a single token to follow a given prompt; we have it roll out a continu‐ ous passage of text by predicting a token, appending that token to the prompt, predicting another token, appending that, and so on. Thinking about the operation of the simulator within this autoregressive loop better matches typical scenarios than thinking about single token prediction, and is thus a better fit to what we typically mean when we talk about GPT. But there's more to this distinction than descriptive point of view. Crucially, the growing sequence of prompt+output text, repeatedly fed back into the loop, preserves information and therefore constitutes state, like the tape of a Turing machine. In the Simulators post, janus writes: I think that implicit type-confusion is common in discourse about GPT. “GPT”, the neural network, the policy that was optimized, is the easier object to point to and say definite things about. But when we talk about “GPT’s” capabilities, impacts, or alignment, we’re usually actually concerned about the behaviors of an algorithm which calls GPT in an autoregressive loop repeatedly writing to some prompt-state... The Semiotic physics post defines the term trajectory to mean the sequence of tokens—prompt plus generated-output-so-far—after each iteration of the autoregressive loop. In semiotic physics, as is common in both popular and technical discourse, by default we talk about GPT as a generator of (linguistic) trajectories, not context-free individual tokens. Simulators are multiverse generators GPT's token-by-token production of a trajectory is stochastic: at each autoregressive step, the trained model generates an output probability distribution over the token vocabulary, samples from that distribution, and appends the sampled token to the growing trajectory. (See the Semiotic physics post for more detail.) Thus, every token in the generated trajectory is a branch point in the sense that other possible paths would be followed given different rolls of the sampling dice. The simulator is a multiverse generator analogous to (both weak and strong versions of) the many-worlds interpretation of quantum mechanics.[2] janus (unpublished) says "GPT is analogous to an indeterministic time evolution operator, sampling is analogous to wave‐ function collapse, and text generated by GPT is analogous to an Everett branch in an implicit multiverse." Semiotic physics It's in this analogical sense that a simulator like GPT implements a "physics" whose "elementary particles" are linguistic tokens. Like real-world physics, the simulator's "physics" leads to emergent phe‐ nomena of immediate significance to human beings. In real-world physics, these emergent phenomena include stars and snails; in semiotic physics, they're the stories the simulators tell and the simulacra that populate them. Insofar as these are unprecedented rhymes with human cognition, they merit investigation for their own sake. Insofar as they're potentially beneficial and/or dangerous on the alignment landscape, understanding them is critical.[3] Texts written by GPT include dynamic representations of extremely com‐ plex, sometimes arguably intelligent entities (simulacra) in contexts such as narrations; these entities have trajectories of their own, distinct from the textual ones they supervene on; they have continuity within contexts that, though bounded, encompass hundreds or thousands of turns of the autoregressive crank; and they often reflect real-world knowledge (as well as fictions, fantasies, fever dreams, and gibberish). They interact with each other and with external human beings.[4] As janus puts it in Simulators: I have updated to think that we will live, however briefly, alongside AI that is not yet foom’d but which has inductively learned a rich enough model of the world that it can simulate time evolution of open-ended rich states, e.g. coherently propagate human behavior embedded in the real world. As linguistically capable creatures, we experience the simulator's outputs as semantic. The tokens in the generated trajectory carry meaning, and serve as semiotic signs. This is why we refer to the simulator's physics- analogue as semiotic physics. In real-world physics, we have formulations such as the Schrödinger equation that capture the time evolution operator of quantum mechanics in a way that allows us to consistently make reliable predictions. We didn't always have this knowledge. janus again: The laws of physics are always fixed, but produce different distrib‐ utions of outcomes when applied to different conditions. Given a sampling of trajectories – examples of situations and the outcomes that actually followed – we can try to infer a common law that gen‐ erated them all. In expectation, the laws of physics are always implicated by trajectories, which (by definition) fairly sample the conditional distribution given by physics. Whatever humans know of the laws of physics governing the evolution of our world has been inferred from sampled trajectories. With respect to models like GPT, we're analogously at the beginning of this process: patiently and directly observing actual generated trajecto‐ ries in the hope of inferring the "forces and laws" that govern the simula‐ tor's production of meaning-laden output.[5] The Semiotic physics post explains this project more fully and gives numerous examples of existing and potential experimental paths. Semiotic physics represents a naturalistic method of exploring the simu‐ lator from the output side that contrasts with and complements other (undoubtedly important) approaches such as "[thinking about] exactly what is in the training data", as Beth Barnes has put it. The semantic realm and the physical realm Simulators like GPT reflect a world of semantic possibilities inferred and extrapolated from human linguistic traces. Their outputs often include remarkable renditions of the real world, but the relationship between what's depicted and real-world physical law is indirect and provisional. GPT is just as happy to simulate Harry Potter casting Expelliarmus as an engineer deploying classical mechanics to construct a suspension bridge. This is a virtue, not a flaw, of the predictive model: human discourse is indeed likely to include both types of narrations; the simulator's output distributions must do the same. Therefore, it's a misconception to think of semiotic physics as approxi‐ mating or converging on real-world physics. The relationship between the two is analogical. Taking a cue from the original Simulators post, which poses the question of self-supervised learning in the limit of modeling power, people some‐ times ask whether the above conclusion breaks down for a sufficiently advanced simulator. At some point, this argument goes, the simulator might be able to minimize predictive loss by modeling the physical world at such a fine level of detail that humans are emulated complete with their cognitive processes. At this point, human linguistic behaviors are faithfully simulated: the simulator doesn’t need to model Harry Potter; it’s simulating the author from the physical ground up. Doesn’t this mean semiotic physics has converged to real-world physics? The answer is no. Leaving aside the question of whether the hypothesized evolution is plausible—this is debatable—the more important point is that even if we stipulate that it is, the conclusion still doesn’t follow, or, more precisely, doesn’t make sense. The hypothesized internalization of real-world physics would be profoundly significant, but unrelated to semiotic physics. The elementary particles and higher-level phenomena are still in disjoint universes of discourse: quarks and bosons, stars and snails (and authors) for real-world physics; tokens, stories, and simulacra for semiotic. Well then, the inquirer may want to ask, hasn’t semiotic physics con‐ verged to triviality? It seems no longer needed or productive if an inter‐ nalized physics explains everything! The answer is no again. To see this, consider a thought experiment in which the predictive behavior of the simulator has converged to perfec‐ tion based on whole-world physical modeling. You are given a huge cor‐ pus of linguistic traces and told that it was produced either by a highly advanced SSL-based simulator or by a human being; you're not told which. In this scenario, what's your account of the language outputs produced? Is it conditional on whether the unknown source was simulator or human? In either case, the actual behaviors behind the corpus are ulti‐ mately, reductively, rooted in the laws of physics—either as internalized by the simulator model or as operational in the real world. Therefore ulti‐ mately, reductively, uselessly, the Schrödinger equation is available as an explanation. In the human case, clearly you can do better: you can take advantage of higher-level theories of semantics that have been proposed and debated for centuries. What then of the simulator case? Must you say that the given corpus is rooted in semantics if the source was human, but Schrödinger if it was a simulator? Part of what has been stipulated in this scenario is a predic‐ tive model that works by simulating human language behaviors, in detail, at the level of cognitive mechanism.[6] Under this assumption, the same higher-level semantic account you used for the human case is available in the simulator case too, and to be preferred over the reductive "only physics" explanation for the same reason. If your corpus was produced by micro-level simulation of human linguistic behavior, it follows that a higher-level semantics resides within the model's emulation of human cognition. In this hypothetical future, that higher-level semantic model is what semiotic physics describes. It has converged not with physics, but with human semantics. 1. ^ I recognize some may not be ready to stipulate that human-style semantics is a necessary component of the simulator's model. I think it is, but won't attempt to defend that in this brief note. Skeptics are invited to treat it as a hypothesis based on the ease and consistency with which GPT-3 can be prompted to produce text humans recognize as richly and densely meaningful, and to see test‐ ing this hypothesis as one of the goals of semiotic physics. 2. ^ It's in the nature of any analogy that the analogues are similar in some ways but not others. In this case, state changes in semiotic physics are many orders of magnitude coarser-grained (relative to the state) than those in quantum physics, the state space itself is infinitesimally smaller, the time evolution operator carries more information and more structure, and so on. We can look for hypotheses where things are similar and take caution where they're different, bearing in mind that the analogy itself is a prompt, not a theory. 3. ^ I don't attempt to explore alignment implications in this post, which is meant simply to introduce the high-level semiotic physics concept. Such issues are touched on in the original Simulators post and its comments. 4. ^ This said, it's worth emphasizing that simulacra need not be human, or animate, or agentic at all. 5. ^ There's no implication or expectation that the time evolution opera‐ tor of semiotic physics will be representable in such a compact form as the Schrödinger equation. The balance of information load between state and time evolution operator in the simulator is very different from the analogous balance in quantum mechanics. In the latter, a relatively simple operator transforms a vast state, while in a GPT-like system, the state is many, many, many orders of magni‐ tude simpler, and the operator—the simulator's trained model— comparatively vast. For its dynamics to be captured in a one-line formula would imply a surprising degree of compressibility. 6. ^ Again, this is dubious. But it must be premised even to arrive at this scenario.