## Transcript ### Start stream **0:28** · e **0:58** · e **1:28** · e **1:58** · e **2:28** · e **2:58** · e **3:28** · e **3:58** · e e ### SW starts talking **4:59** · hello one well I'm going to be talking about something that I posted at the end of last week called what's really going on in machine learning some minimal models it's kind of the result of me being curious about the answer to that question for quite a long time and uh finally having a bit of a breakthrough and understanding it as a result of some work on biological evolution that was kind of a surprise in terms of uh a **5:27** · discovery that I sort of didn't expect to make that I made a few months ago it's kind of led to some ideas about machine learning so let me uh tell you what I said piece that I wrote and then uh after I've done with that I'll uh take **5:43** · some questions and be happy to try and provide answers all right the mystery of machine learning it's surprising how little is known about the foundations of machine learning yes from an engineering point of view an immense amount has been figured out about how to build neural Nets that do all kinds of impressive and sometimes almost magical things but at a fundamental level we still don't really know why neural Nets work and we don't ### The Mystery of Machine Learning **6:09** · have any kind of scientific big picture of what's going on inside them basic structure of neural Nets can be pretty simple but by the time they're all trained up with their weights and so on it's been hard to tell what's going on or even to get any good visualization of it and indeed it's far from Clear **6:26** · even what aspects of the whole setup are actually essential and what are just details that have perhaps been grandfathered all the way from when computational neuron Nets were first invented in the 1940s well what I'm going to try to do here is to get underneath this and to **6:42** · sort of strip things down as much as possible I'm going to explore some very minimal models that among other things are more directly amable to visualization at the outset I wasn't at all sure that these minimal models will be able to reproduce any of the kinds of things we see in machine learning but rather surprisingly it seems that can and uh the uh Simplicity of their **7:03** · construction makes it much easier to kind of see inside them and to get more of a sense of what essential phenomena actually underly machine learning one might have imagined that even through the training of a machine learning even though the training of a machine Learning System might be Securus somehow in the end the system would do what it does through some kind of identifiable and explainable mechanism but what we'll see is that in fact that's typically not at all what happens instead it looks much more as if **7:33** · the training manages to home in on some wild computation that just happens to achieve the right results machine learning it seems isn't a building structured mechanisms rather it's basically just sampling from the typical complexity one sees in the computational universe picking out pieces whose Behavior turn out to overlap what's needed and in a sense therefore the **7:57** · possibility of machine learning is ultimately yet another consequence of my favorite phenomenon of computational irreducibility so why is that well it's only because of computational irreducibility that there's all that richness in the computational universe **8:13** · and more than that it's because of computational irreducibility that things end up being effectively random enough that the Adaptive process of training a machine Learning System can reach success without getting stuck but the presence of computational irreducibility also has another important implic that even though we can expect to find limited pockets of computational reducibility we can't expect kind of a general narrative explanation of what a machine Learning System does in other words there won't be a traditional say **8:44** · mathematical general science of machine learning or for that matter probably also Neuroscience instead the story will be much closer to the fundamentally computational new kind of science that I've explored for so long and that has brought us our physics project and the **9:01** · ruad so in many ways the problem of machine learning is a version of the general problem of adaptive Evolution as encountered for example in biology in biology we typically imagine that we want to adaptively optimize some overall Fitness of a system in machine learning we typically try to adaptively train a system to make it align with certain goals or behaviors most often defined by examples and yes in practice this is often done by Trying to minimize a quantity normally called the L **9:30** · and while in biology there's a general sense that things arise through Evolution quite how this works has always been rather mysterious but rather to my surprise I recently found a very simple model that seems to do well at capturing at least some of the most essential features of biological evolution and while the model isn't the same as what we'll explore here for machine learning it has some definite similarities and in the end we'll find that the core phenomena of machine learning and a biological evolution appear to be remarkably aligned and both fundamentally connected **10:02** · to the phenomenon of computational irreducibility so most of what I'll do here focuses on kind of foundational theoretical questions but in understanding more about what's really going on in machine learning and what's essential and what's not we also be able to begin to see how in practice machine learning might be done differently potentially with more efficiency and more generality so let's start off here talking about traditional neural Nets ### Traditional Neural Nets **10:31** · so kind of to to begin the process of understanding the essence of machine learning let's start from a very traditional and familiar example a fully connected um multi-layer perceptron neural net that's been trained to compute a certain function f of x so let me show you that **10:59** · way here okay um screen around here here we go all right so if one gives a value X top here then **11:20** · after Rippling through the layers of this network one gets a value that almost exactly corresponds to our function f ofx down here at the bottom so if we scan through different inputs uh so okay so this is showing us how if **11:42** · we give a particular value X at the top here as we Ripple through the neural net we'll get a whole sequence of different values and the value we get at the bottom will correspond to the value of f ofx for that input X so if we scan through different inputs X we see different patterns of intermediate values inside the network and uh so uh here on a log scale **12:13** · and linear scale at the top log scale at the bottom here's how these intermediate values that occur in the neural net change with the input X so the input X is on the xaxis here and what we're seeing each curve corresponds to the way that the value of a particular artificial neuron changes in in the neuron net and so the **12:37** · uh uh this red highlighted part is the final output value and we can see that the way that that final output value emerges looks very complicated so how is the neuron that ultimately put together how are the values that uh **12:59** · we were um uh plotting um here determined well we're using the standard setup for a fully connected multi-layer Network each node each artificial neuron in each layer is connected to all nodes on the layer above um and the values kind of flow **13:20** · down from one layer to the next being multiply by the positive negative or negative weight indicated by color in these pictures associated with the connection through which they flow so the value of a given neuron is found by totaling up all of its weighted inputs from the layer before adding a certain bias value for that neuron and then applying to the result a certain nonlinear activation function here reu or ramp which just says if its argument **13:48** · is less than zero be zero otherwise be equal to that argument so what overall function a given neural net will compute is determined by the collection of weights and biases that appear in the neural net along with its overall connection architecture and the activation function it's using the idea of machine learning **14:10** · is to find weights and biases that produce a particular function by adaptively learning from examples of that function typically we might start from a random collection of Weights then successively tweak weights and biases to train the nnet to reproduce the function so let's see how that works so here we're starting at the beginning from just random weights and we get something a random output that really doesn't look like the function we're trying to get it all then we progressively uh train this by giving it **14:43** · more and more rounds of examples until eventually the weights are set up so that we exactly get the function we want so we can get a sense of kind of how this progresses and yes it's complicated by plotting successive changes in individual weights over the course of the training process the spikes that you see at the end come from kind of neutral changes that don't affect overall Behavior so the overall objective in the **15:14** · training is progressively to decrease the loss in our case the average squared difference between the True Values of F ofx and those generated by the neuron net the evolution of the loss defines a learning curve for the neuron net and uh we can you can see that here and uh the **15:31** · sort of downward glitches in this learning curve correspond to points where the noral net in effect made a breakthrough in being able to represent our function better so it's it's important to note that there's typically Randomness injected into neural net training so if one runs the training multiple times one will get different networks and different learning curves every time so here are some different networks that all have successfully leared this function f ofx and here are the **16:00** · different learning curves corresponding to those different networks but what's really going on in neural net training effectively we're finding a way to compile a function least to some approximation into a neural net with a certain number of real value parameters the example here we happen to be using about 100 parameters but what happens if we use a different number of parameters or set up the architecture of our neural net differently so here are a few examples **16:29** · so indicating that for the function we're trying to generate the network we've been using is pretty much the smallest that will work so by the way here's what happens if we change our activation function from reu that's the the sharp ramp a smoother **16:46** · function called L which kind of has a nice curve to it so later we'll talk about what happens when we do machine learning with discrete systems in anticipation of that it's interesting to see what happens if we take a neural of the kind we've discussed here and quantize its weights and biases in discrete levels so the result is that as as recent **17:11** · experience with large scale neuronet has also shown the basic operation of the neuronet does not require precise real numbers but survives even when the numbers are at least somewhat discrete so so we see here that uh we we when the discreteness when the discretization the GR arities **17:30** · 00005 we still successfully reproduced the function it starts to glitch out when we quantize at level 0.5 and it's seriously glitched by the time we're at 0.05 and it's just gone Bonkers by the time we are we're quantizing the weights in steps of 0. **17:48** · five so uh the the um if we can make a sort of 3D rendering of how that works as a function what's called Delta the discreetness scale this is showing how the function kind of survives as a more or less accurate version of the function until Delta increases and at some point it loses it okay ### Simplifying the Topology: Mesh Neural Nets **18:14** · so next we want to talk about simplifying the topology mesh neural Nets so far we've been discussing very traditional neural Nets but to do machine learning do we really need systems that have all those details for example do we really need every neuron on each layer to get an input from every neuron on the previous layer what happens if instead every neuron just gets input from at most two others say with the neurons effectively laid out in a simple mesh well quite surprisingly it turns **18:44** · out that such a network is still perfectly able to generate a function like the one we've been using as our example and one advantage of such a mesh neural net so what I'm showing here is a case where we we've set the the weights here so that we feed in a number X at the top we get out F ofx at the bottom and the F ofx we get out is our function here one advantage of such a meshur onet **19:10** · is that like a cellular automaton its internal behavior Can readily be visualized in a rather direct way so this is sort of how the mesh net generates its output and uh here we are kind of stepping through different input values X and seeing for each input value **19:30** · X this is how the mesh net generates its output so you can see here the outputs are a bunch of zeros and then it goes to one and back to zero here uh corresponding to the function as we feed inputs that go from whatever neg3 or something up to positive three here so uh we can visualize what's going on **19:54** · in the way that I showed before where we look at the values all the different values in in this case the meshnet as a function of the input X still hard to understand what's going on inside and um looking at these intermediate values doesn't really help much um though we can sort of see something happening in these intermediate values at places where the function f ofx has jumps so how do we train a mher on that **20:22** · basically we can use the same procedure as for a fully connected network of the kind we saw above reu activation functions don't seem to work well for mesh Nets so so here we're going to use reu instead um and **20:39** · uh so this is this is showing the training starting from a random meshnet and progressively updating the weights just a thousand rounds of updating gives us a meshnet that correctly reproduces our function so here's the the actual um **20:57** · evolution of the difference in each individual weight during the training process so each line here corresponds to one of the positions in the meshnet and it's showing how the value of the weight associated with that uh evolves through the rounds of training um here are results for the meshnet for different seeds and we can see they look quite different so this is different random choices of how to make changes as **21:26** · we're doing this training um and uh this is the learning curves very similar to the ones that we saw for fully connected networks above um and uh uh actually at the size we're using our mesh neural Nets have about the same number of connections and thus weights as a as a as our example of a fully connected Network that we looked at few moments ago and um uh and we see that if **21:53** · if we try to reduce the size of our mher on that uh it doesn't do so well at at reproducing our function so if we have a really tiny mesh on that this is with an L activation function we'll see that this is what it gets it's pretty for approximation as we increase the size of the meshnet uh we we still see some glitches at this size but by the time we're at this size here we've accurately been able to train to reproduce our **22:21** · function okay so uh let's go on so now we'll talk about making everything discreet a biological evolution analog so mesh neuron Nets simplify the ### Making Everything Discrete: A Biological Evolution Analog **22:37** · topology of neuronet connections but somewhat surprisingly at first it seems as if we can go much further in simplifying the function systems we're using and still successfully do versions of machine learning and in particular we'll find that we can make our systems completely discret the typical methodology of neural net training involves progressively tweaking real value parameters usually using methods based on calculus and on finding derivatives **23:01** · and one might imagine that any successful adaptive process would ultimately have to rely on being able to make arbitrarily small changes of the kind that are possible with real value parameters but in studying simple idealization of of biological evolution I recently found some sort of striking examples where this isn't the case and where completely discrete systems seem able to capture the essence of what's going on so as an example let's consider a **23:29** · three-color cellular automaton the rule is shown over here and uh the behavior one one generates by repeatedly applying that rule starting from a single cell inial condition is here the rule has the property that the pattern it generates from a single cell ini inial condition survives here for exactly 40 steps and then dies out so that every cell becomes White and the important point is that this rule can be found by a discrete adaptive process the **23:58** · idea is to start say from a null Rule and then at each step to randomly change a single outcome out of the 27 in this rule here um to make IE to make sort of a single point mutation in the rule so most such changes will call the cause the lifetime of the pattern to get further from our Target of 40 and these we discard but gradually we can build up **24:23** · kind of beneficial mutations and that through sort of this Progressive adaptation eventually get to our original lifetime 40 rule so we can make a plot of sort of all the **24:40** · attempts that we made that eventually led us to the reach lifetime 14 we can think of these of this sort of progressive Fitness curve as being directly analogous to the loss Curves in machine learning that we saw before so if we make different sequences of mutations will'll get different Paths **25:01** · of adaptive Evolution and different solutions for rules that have lifetime 40 but um there are there sort of two things that are immediately notable about what's going on here first that all **25:19** · these different Paths of adaptive Evolution seem to be using different ideas to reach their goal presumably analogous to the phenomenon of different branches in the tree of life and second that none of them seem to be using a clear mechanical procedure of the kind we might construct through traditional engineering to reach their goal in still they instead they seem to be finding sort of natural complicated behavior that just happens to achieve **25:43** · the goal it's non-trivial of course that this Behavior can achieve a goal like the one we've set here as well as that simple selection based on random inut random point mutations can successfully reach the necessary Behavior but as I talked about elsewhere in connection with biological evolution this is ultimately a story of computational irreducibility particularly in generating diversity both in behavior **26:08** · and in the paths necessary to reach it but okay so how does this model of adaptive Evolution relate to systems like neuron Nets in the standard language of neuron Nets a model is like a discrete analog of a recurrent convolutional Network it's convolutional because at any given step the same rule is applied locally throughout array of elements it's recurrent because in effect data is repeatedly passed through the same rule the kinds of procedures like back **26:36** · propagation typically used to train neuron Nets wouldn't be able to train such a system but it turns out that essentially as a consequence of computational irreducibility the very simple method of successive random mutation can be successful so let me talk about machine learning in discrete ### Machine Learning in Discrete Rule Arrays **26:56** · rules so let's say we want to set up a system like a neural net or at least a mesh neural net but we want it to be completely discreet and I mean born discrete not just discretized from an existing continuous system how can we do this one approach that as it happens I **27:16** · first considered actually in the mid 1980s but never seriously explored is to make what we can call a rule array like in a cellular automatan there's an array of rules but instead of these cells always being updated according to the same rule each cell at each place in the cellular automet and analog of SpaceTime can make a different choice of what rule it will use and although it's **27:40** · a fairly extreme idealization one can potentially imagine that these different rules represent a discrete analog of different local choices of Weights in a MH onet so as a first example let's consider a ruler way in which there are two possible choices of rules both twocc color nearest neighbor cellular automet uh rules 4 and 146 which are respectively Class 2 and class three so a rule a particular rule array **28:07** · is to find by which of these rules is going to be used at each space-time position in the array so here are a few examples in each in each case we're starting from the same single cell initial condition but in each case the rule array has a different arrangement of rule choices with cells running rule four being that teal color and um uh ones running rule 146 being **28:32** · that pinkish color well we can see that different choices of rule array can yield um different uh behaviors but in the spirit of machine learning we can in effect invert this **28:47** · and find a rule array that uh uh will give some particular Behavior we want that's what we want to do if we're going to do machine learning with this so a simple approach is to do the direct analog of what we did in our minimal modeling of biological evolution progressively make random single point mutations here flipping the identity of just one rule in the rule array and then keeping only those mutations that don't make things worse just a **29:26** · second so so as our sample objective let's ask to find a rule array that makes the pattern generated from a single cell using that rule array survive for exactly 50 steps at first it might not seem obvious that we'd be able to find such a rule array but in fact our adaptive um simple adaptive **29:51** · procedure easily manages to do this and um we'll we can see here that um this has an infinite lifetime this has lifetime six and so on and we're gradually filling in we're gradually mutating we're flipping uh kind of rules in the rule array and progressively getting life times that are closer to and eventually make it to exactly 50 **30:18** · steps so we can look at how did that happen and as the dots indicate here many m ations don't lead to longer lifetimes but every so often the Adaptive process has a breakthrough that increases the lifetime eventually reaching exactly **30:38** · 50 so just as in our model of biological evolution different random sequences of mutations lead to different solutions here to the problem of living for exactly 50 steps so here are a bunch of different solutions different rule arrays that all succeed in having the Single Cell initial condition pattern live for exactly 50 steps some of these are in effect Simple Solutions that require only a few mutations like this one here then um but **31:07** · most like most of our examples in biological Evolutions seem more as if they just happen to work effectively by tapping into just the right fairly complex behavior is there a sharp distinction between these cases well looking at the kind of collection of Fitness curves for these examples it doesn't really seems **31:29** · so it's not too difficult to see how to construct a construct a simple solution just by sort of strategically placing a single instance of the second rule in the rule array here it is this will live for a a specified number of steps um but the point is that that adaptive evolution by repeated mutation normally won't discover this simple solution it uh and what's significant is that the Adaptive Evolution could nevertheless still successfully find some solution even though it's not the **32:01** · one that's one that's understandable to us so the cellum rules we've been using so far take three inputs but it turns out we can make things even simpler by just putting ordinary two input BL functions into our rule array for example we can make a rule array by just **32:21** · uh using the functions and an exor those are the R equals a half rules 8 and six so different and an exor rules rule arrays show different Behavior but are there for example and an Exel ruer arrays that will compute any of the 16 possible two input Boolean functions we can't get not or any of the other eight other functions that map kind of uh white white to black or uh **32:51** · false false to true but it turns out we can get all eight functions that don't do that and uh this is this is how we can we can do that we're assuming the sort of the missing inputs are effectively white so **33:11** · uh let's see in fact we can we can also get and an EXO ruler arays to find all other all other even Boolean functions so for example here are ruler arays for the uh for a couple of three input uh **33:28** · rules corresponding to the rule 30 cellular tomaton rule 110 uh cellular tomatan those those are those bullan functions reproduced using and plus X or rule arrays so it might be worth mentioning **33:43** · that the ability to set up such rule arrays is related to functional completeness of the underlying rules we're using that's not quite the same thing functional completeness is about setting up arbitrary formulas that can in effect allow long range connections between intermediate results here all information has to explicitly flow through the array but for example the functional completeness of nand allows it to generate all Boolean functions when combined for example with first uh though sometimes the rule arrays that are needed can be quite **34:18** · large okay but what happens if we try to use our adaptive Evolution process say to solve the problem of finding a pattern that survives for exactly 30 steps STS here's a result for and the exor ruler Ray and uh you can see it overshot here a little bit and then came back and eventually that particular pattern of ANS and exor makes an initial input of a **34:46** · single black Cell live for exactly 30 steps um here are examples of other ruler arrays that again successfully lead to patterns that live for exactly 30 steps um none of which in this case look **35:04** · particularly sort of mechanistic or constructed okay so what about learning our original function f ofx well first we have to decide how we're going to represent the numbers X and F ofx in our discrete rray system one approach is to do this simply in terms of the position of a black cell kind of like one hot encoding so for example uh let's Show an example here in this case where um there's an initial black cell at a position um corresponding to about **35:34** · x = - 1.1 and then the result after passing through the rule array is a black cell that's at the position corresponding to f ofx equals 1 so now the question is whether we can find a rule array that successfully Maps the initial to final cell positions according to the mapping X goes to F ofx that we want well here's an example that at least comes close to doing that uh we **36:02** · we're taking the the array to be cyclic here which is why things sort of come back the other side in some cases and and sometimes we'll see that uh uh it doesn't quite nail the result usually it Nails the result but sometimes it doesn't just doesn't make it at all it's kind of like a partial function it doesn't evaluate doesn't give any value at all here so how did we find this well we **36:25** · just used a simple adaptive algorithm Evolution process in direct analogy to the way it's usually done in machine learning we set up training examples here of the form things like this these are uh values of X that go to values f **36:41** · ofx um then we repeatedly Made Simple point mutations in our rule array keeping those mutations where the total difference from all the training examples didn't increase and after 50,000 mutations we got uh the final result for the rule array we're showing here so we can get some sense of kind of how we got there by showing the sequence of intermediate results uh where we get closer to the goal of reproducing that function uh as as opposed to we're **37:13** · showing we've kept in the in the Adaptive Evolution we keep kind of the neutral uh results where we stay the same performance as before but in this picture we're just showing the cases where the performance improves relative to the previous picture okay so here are the corresponding ruler rays that are um **37:36** · successively uh that correspond to these kind of successive improvements in reproducing the function and in each case we're highlighting those elements of the rule array that changed um from the previous iteration of the rule array the previous version of the rule array and we're also showing in this particular case the computation of f of zero to show what that does and we'll see it's it's doing things it's doing okay back here but obviously other pieces of the of the other values of X **38:06** · weren't doing okay and so the rule array got progressively updated until eventually here they're all doing okay so once again an important takeaway is that we don't see identifiable mechanism in what's going on instead it looks more as if the rule arrays we've got just happen to do the computations we want their behavior is complicated but somehow we manage to tap into it to compute our F ofx but how robust is this computation **38:39** · well a key feature of typical machine learning is that it can generalize away from the specific examples it's been given it's never been clear just how to characterize that generalization when there's an image of a cat in a dog suit start being identified as an image of a dog but at least when we're talking about classification time tasks we can think of what's going on in terms of basins of attraction that lead to attractors corresponding to our classes it's all considerably easier to analyze though in the kind of discrete system we're exploring here for example **39:11** · we can readily enumerate all our training inputs I.E all initial States containing a single black cell and then see how frequently these cause any given cell to be black uh **39:29** · and um so this is kind of uh showing superimposing all of our examples um let's see this is what happens if we make this plot at each of the successive breakthroughs during training this is showing how the kind of the overall set **39:48** · of excitations of the network gradually correspond gradually correspond better to getting the final function so what happens if this is a case where we're looking at sort of valid inputs where we just have a single black cell in the input what about all possible inputs including ones that don't just contain a single black cell well because this is a discrete system we can enumerate all of them and compute the **40:15** · overall frequency for each cell in the array to be black so this is now not just the valid inputs that correspond to a single black cell but all possible inputs so as we might expect the result result is sort of considerably fuzzier than what we got purely with our training inputs but there's still a strong trace of the discrete values of F ofx that appeared in the training data if we plot the overall probability for a given final cell to be black we see here **40:43** · Peaks at the positions corresponding to the value Z and one that F ofx takes on so this is now plotting the probability on logge scale the probability to get a certain output value and we see that the vastly uh vastly overwhelming likelihood **41:02** · is that we'll get one of the output values that correctly corresponds to an output to the the function okay so because our system is discret we can just look at explicitly at all possible outcomes that occur funny quotes at the top here um and uh we can see that **41:24** · uh the most common overall from all possible including meaningless inputs is the meaningless sort of all white state that basically happens when the computation never makes it to the output but the next most common outcomes correspond exactly to f ofx equals z and f ofx equals 1 after **41:45** · that is a kind of superposition outcome where f ofx is in effect both zero and one but okay so what initial states are in the basins of Attraction of I.E will evolve to the various outcomes here the Fairly FL plats plots fairly flat **42:03** · plots uh here indicate that the overall density of black cells gives very little information about what attractor a particular initial state will evolve to so this means we have to look at specific configurations of cells in initial conditions so as an example let's say we start from this initial condition which evolves to this final condition now we can ask what happens if we look at a sequence of slightly different initial conditions so here we're showing **42:35** · kind of in black and white initial conditions that still evolve to the original attractor State and in pink ones that evolve to some different state so well what's actually going on **42:51** · inside here this is all a bit complicated here are a few examples kind of highlighting cells whose values change as a result of changing the initial condition so as is typical in machine **43:06** · learning there doesn't seem to be any simple characterization of the form of the Basin of Attraction but now we have a sense of what the reason for this is it's another consequence of computational irreducibility computational irreducibility gives us the effective Randomness that allows us to find useful results by adaptive Evolution but it also leads to changes having what seem like random and unpredict able results worth noting by **43:29** · the way that we should probably we could probably dramatically improve the robustness of our attractive as by specifically including our training data examples that uh uh including training data examples that have kind of noise injected all right let's go on ### Multiway Mutation Graphs **43:52** · here going to talk now about multi-way mutation graphs so in doing machine learning in practice the goal is typically to find some collection of Weights so on that successfully solve a particular problem but in general there will be many such collections of weights with typical continuous weights and random training steps it's very difficult to see the whole ensemble of possibilities but in our discrete rule array systems this becomes more feasible **44:20** · so let's consider a tiny 2x two rule array with two possible States two possible rules sorry we can make a graph whose edges represent all possible point mutations between those ruer Rays so in our adaptive Evolution process we're always moving around a graph like this but typically most moves **44:44** · will end up in states that are rejected because they increase whatever loss we've defined so consider the problem of generating an and an Plus xor ruay in which we end with lifetime four patterns defining the loss as how far we are from this lifetime we can draw a graph that shows all possible adaptive Evolution paths that always progressively decrease **45:07** · the loss so the result here is a multi-way graph of the type we've now seen in a great many kinds of situations notably our recent study of biological evolution and although this particular example is quite trivial the idea in general is that different parts of such a graph represent different Strat iies for solving a problem and in direct analogy to our physics project and our studies of things like game graphs one can imagine such strategies being laid out in a branchial space defined by **45:38** · Common ancestry of configurations in the multi-ray graph and one can expect that while in some cases the branchial graph will be fairly uniform in other cases it will have quite separated pieces that represent fundamentally different strategies of course the fact that underlying strategies may be different doesn't mean that the over Behavior or performance of the system will be noticeably different and indeed one expects that in most cases computational irreducibility will lead to enough effective Randomness that there'll be no discernable difference but in any case **46:10** · here's an example starting with a rule array that contains both and and exor where we observe distinct branches of adaptive Evolution that lead to different solutions to the somewhat trivial problem of finding a configuration of this rule array with a lifetime of exist exactly for and those Solutions are uh highlighted **46:33** · here okay let's talk about optimizing the learning process how should one actually do learning do the learning in machine learning in Practical work with traditional neuron Nets learning is normally done using systematic algorithmic methods like back propagation but so far all we've done here is something much simpler we've leared by successfully making random point mutations and keeping only ones that don't lead us further from our goal ### Optimizing the Learning Process **47:00** · and yes it's interesting that such a procedure can work at all and as we've discuss elsewhere this is presumably very relevant in understanding phenomena like biological evolution but as we'll see there are more efficient and probably much more efficient methods of doing machine learning even for the kinds of discrete systems that we're studying here so let's uh let's start by looking **47:24** · again at our earlier example of finding an and an EXO rule aray that gives a lifetime of exactly 30 so at each step in our adaptive learning process we make a single point mutation changing a single rule in the rule array keeping the mutation if it doesn't take us further from our goal the mutations gradually accumulate every so often reaching a rule array that gives a lifetime close to closer to 30 so just **47:48** · as as above here's a plot again of the lifetime achieved by success of mutations with these sort of internal red dots here corresponding to rejected mutations so we see a series of plateaus at which mutations are accumulating but not changing the overall Lifetime and between these we see occasional breakthroughs where the lifetime jumps so here are the actual uh uh rule array **48:16** · configurations for these breakthroughs with mutations since the last breakthrough highlighted but in the end the process here is quite wasteful in this example we make a total of 1705 mutations but only 780 of them actually contribute to generating the final rule array all the others are discarded along the way so how can we do better one strategy **48:41** · is to try to figure out at each step which mutation is most likely to make a difference and one way to do that is to try every possible mutation in turn at every step as in multi-way evolution and see what effect each of them has on the ultimate lifetime so from this we can kind of construct a change map in which we give the change of Lifetime associated with the mutation at every particular cell the results will be different for every configuration of a rule array uh of the um IE at every step in the **49:14** · Adaptive Evolution but for example uh here's what they are for a particular for um uh for the particular breakthrough configurations that we were just looking at here um and uh so the **49:36** · elements that are colored gray don't affect the results if they're changed the ones colored red have a positive effect with more intense red being more positive and the ones colored blue have a negative **49:52** · effect so let's start say we start from a random rule array then repeatedly construct the change map and apply the mutation that it implies gives the most positive change in effect at each step following the path of steepest descent to get to the lifetime we want I.E to reduce the loss then the sequence of breakthrough configurations that we get is this so this is in effect a slightly **50:17** · more direct path to a solution than our sequence from Pure single point mutations by the way the particular problem of reaching a certain lifetime has a simple enough structure that this steepest descent method when started from a simple uniform rule array finds a **50:36** · very mechanical if slow path to a solution here it is so what about the problem of learning our F ofx function that we had before once again we can make a change map based on the loss we Define they the results for a series of breakthrough configurations the gray regions of the ones where the changes will be neutral so there's still exploration that can be done without there can be done without **51:03** · affecting the loss the red regions are ones that are in effect locked in and where any changes will be deleterious in terms of loss so what happens in this case if we follow the path of steepest descent always making the change that will be best according to the change map well the results are actually what's this **51:30** · that's um the results turn out to be actually quite unsatisfactory from almost any initial condition system quickly gets stuck never finds any satisfactory solution in effect it it seems that deterministically following the path of steepest descent leads us to a local minimum from which we can't **51:49** · escape so what do we missing in just looking at the change map well the change map as we've constructed it has the limitation that it's separately assessing the effect of each possible individual mutation it doesn't deal with multiple mutations at a time which could well be needed in general if one's going to find the fastest path to success and **52:09** · avoid getting stuck but even in constructing the change map there's already a problem because at least the direct way of computing it scales quite poorly in an N byn rule array we have to check in effect uh the effect of flipping about n s values and for each of these we have to run a whole system altogether taking **52:29** · about end to the fourth operations one has to do this separately for each step in the learning process so how do traditional neural Nets avoid this kind of inefficiency the answer in a sense involves a mathematical trick at least as it's usually presented it's all based on the continuous nature of the weights and values in neural Nets which allow us to use methods from calculus so let's say we have a neural net like this that computes some particular function F ofx like this we **52:59** · can ask how this function changes as we change each of the weights in the network in effect this gives us something like our change map from above but there's an important difference because the weights are continuous we can think about infinitesimal changes to them and then we can ask questions like how does f of x change when we make an infinitesimal change to a particular weight let's say the weight here **53:29** · the or equivalently what is the partial derivative of f with respect to uh with respect to some weight w subbi at a particular point x so the now we we get to use a so so **53:52** · this is what this is showing this is this is a weight w subi and this is the change in the value of f associated with as a function of X associated with a an infinite decimal change each of these different curves corresponds to a slightly different value for w subi for that weight so we get to use a sort of key feature of infinite decimal changes that they can always be thought of as sort of just adding linearly because essentially **54:22** · Epsilon squ can always be ignored compared to Epsilon Epsilon is sufficiently infinitesimally small or in other words we can summarize any infinite tmal change just by giving its direction in weight space IE a vector that says how much of each weight should be infinitesimally changed so if we want **54:42** · to change f ofx infinitesimally as quickly as possible we should go in the direction of steepest def descent defined by all the derivatives of f with respect to to the weights so in machine Lear we're typically trying in effect to set the weight so that the form of f ofx we generate successfully minimizes whatever loss we've defined and we do this by incrementally moving in weight space at **55:07** · every step Computing the direction of steepest descent to know where to go next in practice there all sorts of tricks like Adam that try to optimize the way to do this but how do we efficiently compute the partial derivative of f with respect to to each of these these weights in our case yes **55:24** · we could do the analog of generating pictures like the ones above well sorry how do we how do we efficiently compute this partial derivative with respect to each of the weights we could do the analog of these pictures here separately for each of the weights but it turns out that there's a standard result from calculus that gives us a vastly more efficient procedure that in effect maximally reuses the parts of the computation that we've already done it all starts with the textbook chain rule the derivative of nested I.E composed functions **55:58** · so basically says that the infinite decimal change in the value of the whole chain D of C of B of A of X can be computed as a product of infinitesimal changes associated with each of the links in the chain but the key observation is then that what we get for that we get when we when we get to the computation of the change at a certain points in the chain we've already had to do a lot of the computation we need and so long as we stored those results we always have only an incremental computation to perform so how does this **56:31** · apply to neural Nets well each layer in a neural net is an effect doing a function composition so for example our D of C of B of A of X is like kind of a a trivial neural net here um but what **56:47** · about the weights which after all are what we're trying to find the effect of changing well we can include them explicitly in the function we're Computing like this then we could principle symbolically compute the derivatives with respect to these weights here are derivatives with respect to each of those weights and um for the network here that we had this corresponds to uh the **57:11** · corresponding expression ignoring biases is this where f is the activation function so once again we're dealing with nested functions and once again though it's a bit more intricate in this case the computation of derivatives can be done by incrementally evaluating terms in the chain Rule and in effect using the standard neural net method of back propagation so what about the discrete case are there similar methods we can use there well let's not discuss this in **57:40** · detail here but we'll give some indications of what's likely to be involved as a potentially simpler case that's consider ordinary cellular autometer the analog of our change Maps asks how the value of a particular output cell is affected by changes in other cells or in effect what the partial derivative of the output value is with respect to changes in values of other cells for example let's uh **58:06** · consider kind of the highlighted output cell in this cellular Evolution now we can look at each cell in this array and make a change map based on seeing whether flipping the value of just that cell and then running the cellular tomaton forwards from that point would change the value of the highlighted output cell so this is the the change map that we get in that way the form of the change map is different if we look at different output cells um and uh here by the way are some **58:39** · larger change maps for this and a couple of other cellular automaton rules but is there a way to construct such change Maps incrementally one might have thought that there would immediately be at least for cellular autometer that unlike cases here are **58:55** · fundamentally reversible there will be such a an incremental way to compute the change map but actually such reversibility doesn't seem to help much because although it allows us to backtrack C states of the cellular automon it doesn't allow us to trace the separate effects of individual cells so how about using discrete analoges of derivatives in the chain rule let's for example call the function computed by one step in rule 30 cellular and evolution W of X Y and Z we can **59:23** · think of the partial derivative of this function with respect to X at the point x as representing whether the output of w changes when X is flipped starting from value given I shouldn't have used the the W as the as the symbol for this it confuses it with weights this is this is the actual value of the Sol automon so what we're doing here is um uh we we **59:52** · have the function that's the value of the cellon then we're asking whether there's a change when we change the X which is the left hand cell value so no change is integated as false or white cell while A change is indicated as true or black cell and yes one can either explicitly compute the rule outcomes here and then deduce from them the functional form or one can use symbolic rules to directly deduce the functional form which we're showing over here so **1:00:24** · one can compute a discrete anal L of a derivative for any Boolean function so for example we have strange formulas like this and a formula like that the derivative of xor is not um and we could **1:00:39** · also write that in in this way here the derivative of this explicitly written out rule is another explicitly written out Rule and we have all sorts of other all sorts of other curious relations here that are the discrete analog of calculus uh uh now applied to Boolean **1:00:56** · functions actually here's a table of Boolean derivatives for all two input Boolean functions and there's indeed a whole Boolean calculus M can set up for these kinds of derivatives and in particular there's a direct analog of the chain rule where xn is effectively the equality test does x equal **1:01:20** · y okay but so how do we use this to create our change maps in our simple cellular automon case we can think of our change map as representing how a change in an input cell propagates back to previous cells but if we just try to apply our discret calculus rules we run into a problem different chain rule **1:01:39** · chains can imply different changes in the value of the same cell in the continuous case this path dependence doesn't happen because of the way infinite decimals work but in the discreete case it does and ultimately we're doing a kind of backtracking that can really be represented Faithfully only as a multi-way system though if we **1:01:58** · just want probabilities for example we can consider averaging over branches of the multi-way system and the chain Maps we showed above are effectively the result of thresholding over the multi-way system but despite the appearance of such difficulties in the simple cellular automatan case such methods typically seem to work better in our original more complicated ruay case there's a bunch of **1:02:21** · subtlety associated with the fact that we're finding derivatives not only with respect to the value in the ru array but also with respect to the choice of rules which are the analog of Weights in the continuous case so let's consider an and and exor Rule array here um our loss is **1:02:40** · the number of cells whose values disagree with the row shown at the bottom um with the with this cell here um so now we can construct a change map for this rule array both in a direct forward way by just uh seeing what the **1:02:58** · result is from changing each each value each rule array element and backwards using our discrete derivative methods where we effectively resolve the small amount of multi-way behavior by always picking majority values so this is the result from that by direct forward computation and by this backward computation so the results are similar though in this case not exactly the same um so here are a few other examples **1:03:26** · uh and yes in detail there are essentially always local differences between the results from the forward and backward methods but the backward method like in the case of back propagation in ordinary neuron Nets can be implemented much more efficiently and for purposes of practical machine learning it's actually likely to be perfectly satisfactory especially given that the forward method is itself only providing an approximation to the question of which mutations are best to do so as an **1:03:51** · example here are some results from forward and backward methods for the problem of computing of of learning the function f ofx that we've talked about uh again shown for the Breakthrough configurations that we looked at ### What Can Be Learned? **1:04:14** · above okay let's talk about what can be learned so we've now seen quite a few examples of machine learning in action but a fundamental question we haven't yet addressed is what kind of thing can actually be learned by Machine learning **1:04:30** · even before we get to this there's another question given a particular type of system what kinds of functions can it even represent as a first example let's consider a minimal neural net of essentially this form essentially a single layer perceptron Network and with the ru activation function AKA ramp the **1:04:50** · the first set of weights and the first set of Weights here taken to be one as in this picture here the function computed by a neuron net like this has this mathematical form so turns out with enough weights and biases this form can represent any piecewise linear function so for example this essentially just by moving around ramps and using biases and scaling them using weights so for example if we look at this function here um the this **1:05:21** · uh this is the function that's computed by that um neural that that I just showed here um and and here's how that function is kind of built up from uh by adding in successive ramps associated with indivi in individual intermediate nodes or neurons so it's similarly possible to **1:05:44** · get all smooth functions from activation functions like L and so on things get more complicated if we try and represent functions with more than one argument with a single intermediate layer we can only get sort of pwi hyperplan functions I.E functions that change direction only at linear fault lines but um already **1:06:03** · with two intermediate layers and sufficiently many nodes in each of these layers we can generate any piece-wise function of any number of arguments if we limit the number of nodes then roughly we limit the number of boundaries between different linear regions in the values of the functions but as we increase the number of layers with a given number of nodes we basically increase the number of sides that poonal regions with then the function values can have so here are some examples of uh of what functions we can **1:06:32** · compute these are now functions of of two inputs here let's say X and Y and this is the values f of x and y are shown from these from these networks what happens with the mesh Nets that we looked at before well here are a few random examples um of mesh Nets uh **1:06:52** · showing results that are very similar to shallow fully connected Network works with a comparable total number of nodes so how about our fully discrete ruer arrays what functions can they represent we already saw part of the answer earlier when we generated ruer arrays to represent various Boolean functions it turns out that there's a fairly efficient procedure based on Boolean satisfiability we explicitly finding rule arrays that can represent a given function or determine that no ruer Raya of a given size can do this so **1:07:22** · using this procedure we can find minimal and X Plus X or ruler Ray that represent all even three input B functions that is all Ral 1 cellular automon rules so here they are so it's um it's always possible to specify any n input Boolean function by an array of 2 to the n Bits but um **1:07:50** · uh we can we can see from these pictures that when we compile functions into and an exor rule arrays they can take different numbers of bits I.E different numbers of elements in the rule array so in effect the algorithmic information content of the function varies with the language we're using to represent them so for example the uh in the in the case we're looking at here with three input buan functions the distribution of minimal rule array sizes um is this **1:08:19** · there are some functions that are more difficult to represent as and EXO ruler rays and seem to require like 15 ruler elements and others that are easier this is similar to what happens if we represent Boolean functions as Boolean Expressions saying conju of normal form and count the total number of of unary and binary operations **1:08:41** · used okay so we know that there is in principle an and an exor rule array that will compute any even bullan function so now we can ask whether an Adaptive Evolution process can actually find such a rule array say with a the sequence of single point mutations well if we do such adaptive Evolution with a loss that counts the number of wrong outputs for say rule 254 then here's the sequence of **1:09:07** · successive breakthrough configurations that are that uh are produced so here what we're showing every row shows all the possible inputs and shows the outcomes that we get and so on the final row we get the correct sequence of outcomes that corresponds to the final kind of break through configuration of arranging the rule array and uh earlier on when we've got when we've got very few um exors in the in the rule array it **1:09:36** · um it doesn't do very well at Computing this we need this whole mixture of ANS and exors to do this computation so uh the results here are not as compact as the minimal solution if we go back here we can see somewhere here the minimal solution for rule 254 it's down in the corner there and it's a bit smaller than what we've got got here **1:10:01** · um so uh it what's interesting is that even though it's not as compact as the minimal solution that we could find by exhaustively searching all possibilities it seems to always be possible to at least find some and EXO rule array that solves the problem just by using adaptive Evolution with single point mutations so here a result for a few other um uh Boolean functions showing **1:10:30** · kind of on the top row of each of these showing the sequence of breakthrough configurations and the bottom kind of proving that it works by looking at all the possible inputs so yes not only are all even Boolean **1:10:48** · functions representable in terms of and and EXO ruler Rays they're also learnable in this form just by adaptive evolution single point mutations so in what we were doing before we were looking at how machine learning works with our rule arrays and specific cases like for our F ofx function but now we've got a case where we can explicitly enumerate all possible functions at least of a given class in a **1:11:12** · sense what we're seeing is evidence that machine learning tends to be very Broad and capable at least in principle of learning pretty much any function of course there can be specific restrictions like the and and X or ruler arays we're using here can't represent odd functions where you know white plus white goes to to Black and so on the **1:11:33** · nand plus first rule arrays we discussed above can do that case but in general it seems to be a reflection of the principle of computational equivalence that pretty much any setup is capable of representing any function and also adaptively learning it by the way it's a lot easier to discuss questions about representing or learning any function when when's dealing with discrete countable functions because one can expect to either be able to exactly get a given function or not but continuous **1:12:02** · functions it's much more complicated because one's pretty much inevitably dealing with approximations unless one can use symbolic forms which are basically discret so for example while we can say as we did above that reu neural Nets can represent any piecewise linear function in general will only be able to imagine successively approaching an arbitrary function much like when you kind of uh add progressively more terms in a simple fora **1:12:38** · series okay so looking thinking back to our results for discrete rural Rays one notable observation um is that uh uh um while we can **1:12:55** · successfully reproduce all these different bullan functions the actual rule array configurations that achieve this tend to look quite messy and indeed it's much the same as we've seen throughout throughout here machine learning can find Solutions but they're not structured Solutions they're in effect just solutions that happen to work so are there more structured ways of representing Boolean functions with ruler Ray so here are sort of two **1:13:22** · possible minimum size and the ex all rule arrays that represent rule 30 at um the next larger size there **1:13:39** · are there are more possibilities for rule 30 and uh there are also rule arrays that can represent rule 110 all of these can represent rule 110 but in none of these cases is that obvious structure that allows us to immediately see how these computations work or what function is being computed but what if we try to explicitly construct espe essentially by **1:14:04** · standard engineering methods a rule array that computes a particular function we can start by taking something like the function for rule 30 and writing it in terms of and and exor I.E in ANF for algebraic normal form so this is that's the rule 30 function in ANF we can imagine implementing this using an evaluation graph here it is and but now it's it's easy to turn this into a rule array and yes uh we haven't gone **1:14:35** · all the way here and arranged to do things like copy inputs which is what those previous Ru arrays did manage to do so evaluating this rule array for different inputs we find that it indeed gives rule 30 um doing the same thing for rule **1:14:53** · 110 uh the the andex or expression is this the evaluation graph is this and the rule array is this and at least with the evaluation graph as a guide we can readily sort of see what's happening here but the rule array we're using is considerably larger than our minimal solution solutions that we had before or **1:15:18** · even than the solutions we found by adaptive Evolution and this doesn't even solve the problem of copying initial variables and so on so it's a typical situation that one sees in many other kinds of systems like for example sorting networks it's possible to have a constructed solution that has clear structure and regularity and is understandable but minimal Solutions all ones found by adaptive Evolution tend to be much smaller but they also almost always look in many ways random and aren't readily understandable or **1:15:48** · interpretable so so far we've been looking at rule arrays that compute specific functions but in getting a sense of what rule Ru arrays can do we can consider rule arrays that are programmable in that their their input specifies what function they should compute so here for example is an and an exor rule array um found by adaptive Evolution that takes the bit pattern of any even Boolean function as input **1:16:18** · should have said on the right here then applies that Boolean function to the inputs on the left and um with this uh with this rule array we can now compute any possible even buan function so here's this rule array uh evaluating or given the Rule table effectively for or feder's input ### Other Kinds of Models and Setups **1:16:56** · okay let me talk about other kinds of models and setups so our general goal here has been to set up models that capture most essential features of neural Nets and machine learning but that are simple enough in their structure that we can readily look inside and get a sense of what they're doing mostly we've concentrated on ruler Rays as a way to provide a minimal analog of standard sort of perceptron style feedforward neuron Nets what about other architectures and setups **1:17:25** · well in effect our rule arrays are space-time IM inhomogeneous generalizations of cellular autometer in which adaptive Evolution determines which rule say from a finite set should be used at every spatial position and every time step a different idealization in fact we already used uh above before **1:17:46** · is to have an ordinary homogeneous cellular automaton but with a single Global rule determined by adaptive evolution so rule arrays are the analog of feedforward networks in which a given rule in the rule array is in effect used only once as data flows through the system ordinary homogeneous cellular autometer are like recurrent networks in which a single stream of data is in effect subjected over and over again to the same rule there are various **1:18:15** · interpolations between these cases for example we can imagine a layered rule array that uh in which rules different steps can be different but those on a given step are all the same so a system like this can be viewed as an idealization of a convolutional neural net in which a given layer applies the same kernel to all elements at all positions but different layers can apply different kernels a layered rule can't **1:18:40** · encode as much information rule array rather it can't encode as much information as a general rule array but it's still able to show machine learning style phenomena and so here for example it's there's an Adaptive evolution of a layered and plus X or rule array that progressively solves the problem of generating a pattern that lives for exactly 30 steps **1:19:04** · oops so one could also Imagine vertically layered rule arrays in which different rules are used at different positions but any given position keeps running the same rule forever however at least for the kinds of problems we've considered here it doesn't seem sufficient to just be able to pick the positions at which different rules are run one seems to either need to change rules at different time steps or one needs to be able to adaptively evolve the underlying rules **1:19:35** · themselves okay so ruer arrays and ordinary cellular autometer share the feature that the value of each cell depends only on the values of neighboring cells on the step before but in neural Nets it's standard of the value at a given node to depend on the values of lots of nodes on the layer before and what this makes this straightforward in neural Nets is that weighted and perhaps otherwise transformed values from previous nodes are taken to be combined by simple numerical addition and addition being **1:20:03** · Nan associative can take any number of inputs in a cellular automet or Boolean function there's always a definite number of inputs determined by the structure of the function the most straightforward case the inputs come only from nearest neighboring cells but there's no requirement that this is how things need to work and for example we can pick any local template to bring in the input for our function this template could either be the same at every position at every step or it could be picked from a different set differently **1:20:33** · at different positions in effect giving us template arrays as well as rule arrays so what about having a fully connected Network as we did in our first neural net examples Above So to set up a discrete analog of this we first need some kind of discrete ener associative accumulator function to fill the place of numerical addition and for this we could pick a function like and or exor **1:20:56** · or or or majority and if we're not just going to end up with the same value at each node on a given layer we need to set up some analog of a weight associated with each connection which we can achieve by applying either Identity or not I flip or not to the value flowing through each connection so here's an example of this kind of network trained in this case to compute our function f ofx there are just two kinds of connections here flip which is I think blue or don't flip **1:21:28** · which is I think red and at each node we're Computing the majority function in other words giving one if the majority of the inputs are one and zero otherwise so with the one hot encoding of input and output that we used before for our function f ofx uh here are some examples of how this network evaluates our function so **1:21:52** · remembering that the value of x is determined by which cell was black on the initial State and the value of f ofx by which cell is black on the final state so uh so this particular setup was trained **1:22:12** · using just a thousand steps of single point mutation applied to the connection types the loss systematically goes down but the configuration of the connection types continues to look quite random even as it achieves zero loss I even after the function has been completely learned so this is a picture of um uh **1:22:32** · this is a picture of the actual values of the um uh kind of the weights the flip or no flip and that's a picture of the loss as a function of number of uh of rounds of training so in in what we've done so far we **1:22:50** · assume that all connections continue to be present though their types are effectively signs can change but we can also consider a network where connections can end up being zeroed out during training so they're effectively no longer present okay but much of what we've done **1:23:07** · here with machine learning has centered around trying to learn transformations of the form X goes to fex but another typical application of machine learning is autoencoding or in effect learning how to compress data representing a certain set of examples and once again it's possible to do such a task using rule arrays with learning achieved by a series of single point mutations so as a starting point consider training a rule array of cellular treton rules 4 and 146 to **1:23:34** · produce unchanged reproduce unchanged a block of black cells of any width so one might have thought that this would be trivial but it's not because in effect the initial data inevitably gets ground up inside the ruer array and has to be reconstituted at the end but **1:23:53** · nevertheless it's possible to train a ruler a to at least roughly do this so we're seeing you know the block the Black Block at the beginning is more or less reconstituted at the End by going through this the same rule array in all cases um but the rule arrays that do this once again um uh uh end up looking **1:24:15** · quite quite random set to set up a non-trivial autoencoder let's imagine that we progressively squeeze this array in the middle creating an increasingly narrow bottleneck through which the data has to flow at the bottleneck we effectively have a compressed version of the original data and we find that at least down to some width of bottleneck um it's possible to create ruler arrays that with reasonable probability connect to successful autoencoders of the original data so the success of llms has **1:24:48** · highlighted the use of machine learning for sequence continuation and the effectiveness of Transformers for this just with other neuron Nets the forms of Transformers that are used in practice are typically very complicated but can one find a minimal model that neveress captures the essence of Transformers let's say we have a sequence we want to continue like this we want to encode each possible value here by a vector as in something like this um so that for example our this **1:25:22** · this is how our initial sequence let me see do I have one here yes this is how our initial sequence is encoded in terms of these vectors here that represent particular values effectively of f ofx um so then let's say we have a head that **1:25:40** · reads a block of consecutive vectors picking off certain values feeding pairs of them into and and exor functions to get a vector Boolean values so here's here's how that works we've got this kind of head that's sampling our blocks of vectors in particular places then feeding those sampled values you can see that that white value getting fed in here to this **1:26:06** · is that an and an x i forgotten to this to this and let's say actually should have made on the on the outside here we should have shown what the outputs for oh no we do we're showing this is an output this is right this is this is um uh this is a one output and this is a uh **1:26:25** · sorry this is a zero output this is a one output this sample these values here and here brought them together and this is a darkened uh kind of reddish uh thing representing a one from that uh from that exor there oh no that that's that's an and I think um actually we can **1:26:47** · see whether it's an and or an exor and it's it's clearly an exor because it's got one black and one white anyway way so ultimately this head is kind of going to slide along our sequence predicting what the next element in the sequence will be but somehow we have to go from our Vector of Boolean values here to probabilities of sequence **1:27:10** · elements potentially we might be able to do this with just with a rule array but for our purposes here we'll use a fully connected single layer identity Plus notot network in which each output node we just find the sum of the number of values that come to it and treat this as determining through a soft Max the probability of the corresponding element so this is the this is how that works for that we're taking those values coming out of those exors and ands and **1:27:39** · we're putting them through this fully connected Network and then we're saying you know what's the um uh what's what's the winning value here which in this case would be the six here um and and uh **1:27:57** · okay so the maximum value is this this maximum value here is a six but this occurs for element five so kind of at zero temperature this will be our best prediction for the next element in the predicted by the system **1:28:19** · so to train this whole system we just make a sequence of random point mutations to everything keeping mutations that don't increase the loss where the loss is basically the difference between the predicted next values and actual next values or more precisely the categorical cross entropy so here's how this loss progresses in a typical neural net training sorry in a typical in a typical **1:28:43** · training this is not a neur that this is a very discretized analog of of a of a neural net so at the end of the training here are the components in our sort of minimal Transformer it's it's a very artist's impression of a Transformer cartoon of a transformer so first come the encodings of the different possible elements in the sequence then there's the head and this is shown applied to the **1:29:16** · encoding of the first elements in the original sequence from there there's a a single layer discrete Network that takes the output from the head and deduces relative probabilities for over over here sorry there's a this is a single layer discrete Network that reduces relative probabilities that's based on sort of summing up how many uh how many flip versus non-flip connections there were to ones versus zeros here um and uh it deduces relative **1:29:47** · probabilities for different elements in this case the um uh the highest probability prediction for the next element is that it should be Element no it shouldn't be element 10 this is that's that's uh that's element one two 3 four five we should label these with BL on the right um and **1:30:14** · uh um let's see so this is um uh yeah it should be element six because we're going here the bottom so that's the that's the highest value from the neural net and the position of that highest value is position six so that's the prediction for what the next element should be of our sequence so to do the analog of an llm **1:30:38** · we start from some initial prompt IE an initial sequence that fits within the width the context window of the head then we progressively apply our kind of minimal transformer for example at each step taking the next element to be the one with the highest prediced probability I.E operating at zero temperature so with this setup the collection of prediction strengths is shown in Gray here with the best **1:31:04** · prediction shown in Red so we're winners here um running this even far beyond our original training data that was our original training data there we see that we get a prediction of the continued sine wave that's our highest probability **1:31:20** · rediction well as we might expect the fact that our minimal Transformer can make such a plausible prediction relies on the Simplicity of our sign Cod if we use more complicated training data like a mathematical mathematically defined sinx plus sin2x uh uh curve here let see um sure oh that's **1:31:45** · that's sort of an that's a continuation of the sign go that's great um but this is now a our sin x plus sin < t2x showing the part we're going to train on and and and the part that we want to predict in blue at the beginning and uh yellow at the end so the result of training and running our sort of minimal Transformer is now this that's um so we see that in **1:32:14** · the region where it's been trained it um it it had training data it does just fine in the region where it's kind of predicting for itself it's really crummy um and not surprisingly it sort of can't figure out the computation to correctly continue the curve um by the way different training runs will involve uh different sequences **1:32:40** · of mutations will leave different predictions sometimes it'll have a kind of periodic hallucination like in this case here okay so in looking at kind of perceptron style neural Nets we wound up using Ru arrays or in effect SpaceTime **1:33:00** · in homogeneous cutometer as our minimal models here we've ended up with a slightly more complicated minimal model for Transformer neural Nets but if we were to simplify it further we would end up with not with something like a solid automaton but instead with something like a tag system in which one has a sequence of elements and at each step removes a block from the beginning and depending on the form that one has adds a block to the end as shown with these these rules down here **1:33:30** · um and so with even a very simple system like this one can generate extremely complex Behavior reinforcing the idea that we've repeatedly seen here that machine learning works by selecting complexity that aligns with the goals that have been set and along these lines one can consider all sorts of of um uh **1:33:53** · different computational systems as foundations of for machine learning here we've been looking at cellular automatan like and tag system like examples but for example our physics project has shown us the power and flexibility of systems based on hypography writing and from what we've seen here it seems very plausible that something like hypography writing can serve as a yet more powerful and flexible substrate for machine learning ### So in the End, What's Really Going On in Machine Learning? **1:34:27** · okay so in the end what's going on in machine learning there are I think several quite striking conclusions from what we've been able to do here the first is just that models much simpler than traditional neural Nets seem capable of capturing essential features of machine learning and indeed these models May well be the basis for a new generation of practical machine learning but from a **1:34:53** · scientific point of view one of the things that's important about these models is that they're simple enough in structure and that it's immediately possible to produce visualizations of what they're doing inside and studying these visualizations the most immediately striking feature is how complicated they look it could have been **1:35:09** · that machine learning would somehow crack systems and find simple representations for what they do but that doesn't seem to be what's going on at all instead what seems to be happening is that machine learning is in a sense just hitching a ride on the general rich of the computational universe it's not specifically building up Behavior one needs rather it's doing what it's doing is to harness Behavior that's already out there in the computational **1:35:39** · universe so the fact that this could possibly work relies on the crucial and at first unexpected fact that in the computational universe even very simple programs can ubiquitously produce all sorts of comp complex Behavior and the point is then that this Behavior has **1:35:57** · enough richness and diversity that it's possible to find instances of it that align with machine learning objectives once's defined in some sense what machine learning is doing is to mine the computational universe for programs that do what one wants and it's not that machine learning Nails a specific precise program rather **1:36:17** · it's that in typical successful applications of machine learning there are lots of programs that do more or less the right thing if what one's trying to do involves something computationally irreducible machine learning won't typically be able to get well enough aligned to correctly get through all the steps of the irreducible computation but it seems that many humanlike tasks that are the particular Focus modern machine learning can successfully be done and by the way **1:36:44** · one can expect that with the minimal models explored here it becomes more feasible to get a real characterization of What kinds of objectives can successfully be achieved by machine learning and what cannot critical to the operation of machine learning is not only that there exist programs that can do particular kinds of things but also that they can realistically be found by adaptive Evolution processes so in what **1:37:07** · we've done here we've often used what's essentially the very simplest possible process for adaptive Evolution a sequence of point mutations what we've discovered is that even this is usually sufficient to lead us to satisfactory machine learning Solutions it could be that our pars of adaptive Evolution would always be getting stuck and not reaching any solution but the fact that this doesn't happen seems critically connected to the computational irreducibility that's ubiquitous in the systems we're studying and that leads to effective Randomness that with overwhelming probability will give us a **1:37:39** · way out of anywhere where we got stuck in some sense computational irreducibility levels the playing field for different processes of adaptive Evolution and lets even simple ones be successful something similar seems to happen for the whole framework we're using any of a wide class of systems **1:37:59** · seem capable of successful machine learning even if they don't have the detailed structure of traditional neuron Nets we can see this as a typical reflection of the principle of computational equivalence that even though systems May differ in their details they're ultimately all equivalent in the computations that they can do the phenomenon of computational irreducibility leads to a fundamental trade-off of particular importance in thinking about things like AI if we want **1:38:24** · to know in advance to be able to know in advance and broadly guarantee what a system is going to be able to do we have to set the system up to be computationally reducible but if we want the system to be able to make the richest use of computation it'll inevitably be capable of computationally irreducible behavior and it's the same **1:38:41** · story with machine learning if we want machine learning to be able to do the best it can and perhaps give us the impression of achieving magic then we have to allow it to show computational irreducibility and if we want machine learning to be understandable it has to be computationally reducible and not able to access the full power of **1:39:02** · computation at the outset though it's not obvious whether machine learning actually has access to such power it could be that there are computationally reducible ways to solve the kinds of problems we want to use machine learning to solve but what we've discovered here is that even in solving very simple problems the Adaptive Evolution process that it's a that that's at the heart of machine learning will end up sampling and using what we can expect to be computationally irreducible processes **1:39:30** · like biological evolution machine learning is fundamentally about finding things that work without the constraint of understandability that's forced in us when we as humans explicitly engineer things step by step could one imagine constraining machine learning to make things understandable to do so would effectively prevent machine learning from having access to the power of computationally irreducible processes and from the evidence here it seems unlikely that with this constraint the **1:39:58** · kind of successes we've seen in machine learning will be possible so what does this mean for the science of machine learning one might have hoped that one would be able to look inside machine learning systems and get detailed narrative explanations for what's going on then in effect would be able to explain the mechanism for everything but what we've seen here suggests that in general nothing like this will work all one will be able to **1:40:23** · say is that somewhere out there in the computational universe there's some typically computationally irreducible process that quotes happens to be aligned with what we want yes we can make General statements strongly based on computational irreducibility about things like the findability of such processes Say by adaptive Evolution but **1:40:42** · if we ask how in detail does the system work there won't be much of an answer to that of course we can trace all its computational steps and see that it behaves in a certain way but we can't expect what amounts to a global human level explanation what it's doing rather **1:40:57** · will basically just be reduced to looking at some computationally irreducible process and observing that it happens to work and we won't have a high level explanation of why but there is one important loophole to this within any computationally irreducible system there are always inevitably pockets of computational reducibility and as I've discussed at length particularly in connection with our physics project it's it's these pockets of reducibility that allow computationally bounded observers like us to identify things like laws of **1:41:29** · nature from which we can build human level narratives so what about machine learning what pockets of computation reducibility show up there from which we might build human level scientific laws much as we the observe emergence of **1:41:45** · simple Continuum behavior from computationally irreducible processes happening at the level of molecules in a gas or ultimate discrete elements of space we can expect that at least certain computationally reducible features will be more obvious when one's dealing with larger numbers of components and indeed in sufficiently large machine Learning Systems it's routine to see smooth curves and apparent regularity when one's looking at the kind of aggregated behavior that's probed by things like training curves but the question about pockets of **1:42:15** · reducibility is always whether they end up being aligned with things we consider interesting or useful yes it could be that machine Learning Systems would exhibit some kind of collective EEG like Behavior but what's not clear is whether this Behavior will tell us anything about the actual information processing or whatever that's going on in the system and if there is to be a science **1:42:35** · of machine learning what we have to hope for is that we can find in machine Learning Systems pockets of computational reducibility that are aligned with things we can measure and care about so given what we've been able to explore here about the foundations of machine learning what can we say about the ultimate power of machine Learning Systems a key observation has been that **1:42:57** · machine learning works by packing on computational irreducibility in effect by finding natural pieces of computational irreducibility that happen to fit with the objectives one has but what if those objectives involve computational irreducibility as they often do when when dealing with a process that's been successfully formalized in computational terms as in **1:43:18** · math exact science computational X and so on well it's not enough that our machine learning system uses some piece of computational irreducibility inside to achieve a particular computational irreducible objective the system would have to do something closely aligned with that actual specific objective it has to be said however that **1:43:40** · by laying be more of the essence of machine learning here it becomes easier to at least Define the issues of merging typical formal computation with machine learning traditionally there's been a trade-off between the computational power of a system and its stainability and indeed in terms of what we seen here this seems to reflect the sense that larger chunks of computational irreducibility are more difficult to fit into something one's incrementally building up by a process of adaptive **1:44:08** · Evolution so how should we ultimately think of machine learning in effect its power comes from leveraging the natural resource of computational irreducibility but when it when it uses computational irreducibility it does so by foraging pieces that happen to advance its objectives imagine one's building a wall **1:44:29** · one possibility is to Fashion bricks of a particular shape that one knows will fit together but another is just to look at Stones one sees lying around and then build the wall by fitting these together as best one can and if one then asks why **1:44:44** · does the wall have such and such a shape the answer will end up basically being because that's what one gets from the stones that happen to be lying around there's no overarching Theory to it in itself it's just a reflection of the resources that were out with there or in the case of machine learning one can expect that what one sees will be to a large extent a reflection of the raw characteristics of computational irreducibility in other words the foundations of machine learning are as much as anything rooted in the science of rology and it's in a sense in large **1:45:16** · measure it's it's in large measure to that science that we should look in our efforts to understand more about about what's really going on in machine learning and quite possibly also in Neuroscience so in what I wrote I also included some historical and personal notes I'll I'll just go through these uh uh and then we'll uh take questions and so on so in some ways it seems to be seems ### Historical & Personal Notes **1:45:47** · like a quirk of intellectual history that the kinds of foundational questions I've been discussing here weren't already addressed long ago in some ways it seems like an inexorable consequence of the only rather recent development of certain intuitions and tools the idea that the brain is fundamentally made of connected nerve cells was considered the latter part of the 19th century took hold in the first Decades of the 20th century with the formalized concept of a neuron net that operates in a computational way emerging in full form **1:46:14** · in the work of Warren McCulla and Walter pits in 1943 so by the late 1950s there were Hardware implementations of neural Nets typically for image processing in the form of perceptrons but despite early enthusiasm practical results were mixed and by the end of the 1960s it was an it was announced that it was uh announced **1:46:35** · that simple cases IM meable to mathematical analysis have been solved leading to the general belief that neural Nets couldn't do anything interesting so ever since the 1940s there had been a trickle of general analyses of neuron Nets particularly using methods from physics but typically these analyses ended up with things like Continuum approximations that could say little about the information processing aspects of neuron Nets meanwhile there **1:47:00** · was an ongoing undercurrent of belief that somehow neuron Nets would both explain and reproduce how the brain works but no methods seemed to exist to say quite how then at the beginning of the 1980s there was a Resurgence of interest in neuron Nets coming from several directions some of what was done concentrated on very practical efforts to get neural networks to do particular human-like tasks but some was more theoretical typically using methods from statistical physics or dynamical systems before long however **1:47:30** · the buzz died down and for several decades only a few groups were left working with neuron Nets then in 2011 came a surprise breakthrough in using neuron nets for image analysis it was an important practical advance but it was driven by technological ideas and development not any significant new theoretical analysis or framework and **1:47:50** · this was also the pattern for almost all of what followed people spent great effort to come up with neuronet systems that worked and all sorts of folklore grew up about how this should best be done but there wasn't really even an attempt at an underlying Theory this was a domain of engineering practice not basic science and it was in this tradition that chat gbt burst onto the scene in late 2022 almost everything about LM seemed to be complicated yes **1:48:16** · there were emperically some large scale regularities like stealing laws and I quickly suspected that the success of llms was a strong hint of General regularities in human language that hadn't been clearly identified before but beyond a few outlier examples almost nothing about what's going on inside llms has seemed easy to decode and **1:48:35** · efforts to sort of put strong guard rails on the operation of the system in effect so as to make it in some way predictable or understandable typically seem to substantially decrease its power a point that now makes sense in the context of computational irreducibility well my own interaction with machine learning in neural net began in 1980 when I was developing my S **1:48:56** · SMP symbolic computation system and wondering whether it might be possible to generalize the symbolic pattern matching foundations of the system to some kind of fuzzy pattern matching that will be closer to human thinking I was aware of neuron Nets but thought of them as sort of semi-realistic models of brains not for example as potential sources of algorithms of the kind I imagined might solve fuzzy matching it **1:49:18** · was partly a result of trying to understand the essence of system like learn that that in 1981 I came up with what I later learned could be thought of as onedimensional cellular autometer so soon I was deeply involved in studying cellular autometer developing new intuition about complex Behavior could arise even in in from simple rules but when I learned about recent efforts to make idealized models uh from of of **1:49:45** · neuron Nets using ideas from statistical mechanics I was at least curious enough to set up simulations try try to understand more about these models but what I did wasn't a success I could neither get the models to do anything of significant practical interest nor did I manage to derrive any good theoretical understanding of them I kept wondering though uh what relationship there might be between cellular autometer that just run and systems like neuron Nets that can also learn in fact in 1985 I tried **1:50:14** · to make a minimal cellular automatan based model to explore this it was what I'm now calling a vertically layered rule array and while in many ways I was already asking the right questions this was an unfortunate specific choice of systems and my experiments on it didn't **1:50:30** · reveal the kinds of phenomena we're now seeing well years went by I wrote a section on human thinking in my book a new kind of science that discussed the possibility of simple foundational rules for the essence of thinking and even included a minimal discreete analog of a neural net at the time though I didn't develop these ideas by 2017 though 15 **1:50:51** · years after the book was published knowing about the breakthroughs and deep learning I began to think I began to think uh more concretely about neuron Nets as getting their power by sampling programs from across the computational universe but I still I didn't quite see how this would work meanwhile there was **1:51:08** · a new intuition emerging from practical experience with machine learning that if you bashed almost any system hard enough it would learn did that mean that perhaps one didn't need all the details of neuron Nets to successfully do machine learning and could one perhaps make a system whose structure was simple enough that its operation would for example be accessible to visualization I particularly wondered about this when I was writing it exposition of chat GPT and llms in early 2023 and I kept talking about llm science but didn't have much of a chance to work on it then few months ago as **1:51:42** · part of an effort to understand the relation between what science does and what AI does I try to kind of throw away experiment which to my considerable surprise seemed to successfully capture some of the essence of what makes biological evolution possible but but **1:51:58** · what about other adaptive evolution in particular machine learning the models that seemed to be needed were embarrassingly close to what I'd studied in 1985 but now I had a new intuition and thanks to wol from language vastly better tools and the result has been my effort here so of course it's only a beginning **1:52:18** · but I'm excited to be able to see what I consider to be the beginnings of foundational science science around machine learning already there are clear directions for practical applications which I'm hoping to explore and there are signs that perhaps we may finally be able to understand just why and when the magic of machine learning works thank you all right I see there are a whole bunch of questions here so let me see what I can address um ### Q&A **1:52:56** · first question from JH how does adding noise help a neuronet approximate a function uh I don't think it does I think it helps in training finding in being able to find a uh a version of the weights for the neural net that will successfully uh represent the function I think the issue is that um in uh as you try and follow **1:53:30** · sort of a steepest descent like path with back propagation and so on there's a tendency to get stuck and if you add Randomness you kind of Wiggle yourself enough that you don't just get stuck in local Minima uh question here about can machine learning find patents in animal language let's not talk about that here but let's will let's talk about that in another um **1:53:58** · in a a Q&amp;A about about science live stream um let's see Perpetual asks could you evolve a discrete system to read the mnist data set that's a a standard data set of um uh of handwritten digits the **1:54:18** · answer is yes we've done it uh coming soon um it works it works very nicely uh Kappa asks um let's see how would the system know if the pattern generated has been learned how does it know what to learn I mean that's that's the challenge for for **1:54:46** · machine learning as for Critter it's like we are we learn things based on what's out there in the world um we it's an interesting sort of philosophical question if we were just sort of disembodied brains you know some strange brain organized in a box could we learn just by purely pondering things that's an interesting philosophical question but for the practicalities of machine learning it's learning from things that are imposed from the outside **1:55:20** · world let's see uh Valentine asks how does this relate to work done by uh someone named Ian Wright using **1:55:40** · maatic or language I do not know that work sounds interesting like to find out about it um let's **1:55:55** · uh Valentin also asks in general one had an efficient way to find the smallest bull inpr that corresponds to a particular truth table wouldn't that be equivalent to having found a polinomial Time solution for the general satisfiability problem yes but the remarkable thing is **1:56:12** · that the uh even the old Davis pum algorithm from the 1950s for finding satisfiable solutions to bullan Expressions is remarkably efficient in many practical cases yes in principle it can uh kind of have a hard **1:56:31** · time and take exponentially long to um uh to find um the uh uh to find out satisfiability for a particular a Boolean expression stated in a particular way in terms of of ANS and ores and so on but in practice I I've **1:56:48** · just been amazed at how efficiently this algorithm Works in lots of practical situations been using it a bunch in solving tiling problems recently for example which is very far from what it was sort of natively intended for and and it works great although there are bad cases in principle where it doesn't **1:57:07** · work but that's the question of sort of what can you learn and so on and the the issue is in sort of practical cases how well does it work but I I was only using that here as kind of a backup as a way to actually uh yet um uh as as a way to **1:57:25** · um um uh the um uh as a as a as a way to kind of as an alternative to exhaustively searching to find the optimal rule arrays that compute something I was not using that as part of the kind of machine learning like Loop here um let's see **1:57:55** · uh OBS asks I wonder if I can use these rule arrays instead of input gate to get gate and forward Gates and LM network instead of using floating Point numbers I I I bet um uh the um the answer is yes um uh we didn't try that yet but um let **1:58:16** · see uh see I those are all the questions we had so I'm happy there any more questions I'm happy to address them for a couple minutes otherwise um thanks for watching **1:58:40** · and um stay tuned for other interesting things happening as we attempt to kind of understand more about the foundations of of machine learning and uh to use the models that we built here these kind of minimal models as kind of seeds for **1:58:57** · practical kinds of uh approaches to to machine learning um oh I see I see a lot of other oh my gosh wait what happened help there were a lot of other questions I was being tricked here I thought I was going to get to go and have my dinner oh wow um **1:59:25** · this seems to be a question from an llm I'm not going to not going to I'm doing human only questions here um there's a comment from Gregory um that this might lead to efficient ways of building improving neural machine Learning Systems yes absolutely uh um **1:59:56** · oh gosh could this analysis benefit from deeper engagement with existing machine learning theory and literature on N Net Theory statistical learning theory optimization possibly yes I consider myself uh medium educated but not **2:00:15** · perfectly educated uh by now there are probably a significant fraction of a million p papers that have been written about those kinds of topics and I certainly don't know what's in all of them uh I've been following this field for 40 years but um haven't um certainly haven't um are not aware of all of the um all of the corners of of What's um uh what's what's being done **2:00:43** · um let's see a bunch of questions here about um one from Muhammad asking about some uh uh whether one can take the things we've done here on sort of minimal models and translate them into real world machine learning uh yeah we've been starting to do that it it it actually seems to be working remarkably well I mean I think the the big takeaway **2:01:08** · is the details don't matter much and that's sort of been the takeaway that that one could take the details don't matter but engineering tricks can help you get there much faster and we don't know the engineering tricks for this kind of approach yet although perhaps we can uh sort of piggyback on the engineering tricks that have been found for traditional uh you know n on that **2:01:32** · continuous machine learning uh let's see um there's a comment here from Roger saying she learning concluding that this is basically machine learning is basically a fine grain curve fitting yeah I mean ultimately that's always true there are no modeless models of anything ultimately there is a a a **2:02:02** · structure of model that is the thing whose parameters you're tweaking to be able to represent whatever it is you're interested in and and that's uh that's something I think that is interesting here is when you do sort of traditional **2:02:18** · curve fitting let's say you've got cubic splines or something to a curve you've got very understandable little you know curvy bits that you're fitting together to make the curve what this is saying is that with neural Nets the pieces you've got that you fit together to make things work are these weirdly shaped kind of random rocks you picked up off the ground type thing they're not sort of smoothly brick like kinds of things **2:02:44** · they're these weird shapes which just happen to fit together rather than being things which for example we can understand oh this piece fits with that that piece fits with that piece um let's see **2:03:14** · uh adjusting meta question here which I I don't know um well okay um let's see questions from question from Kane here about hybrid AI architectures um yeah I mean as a in my day job we've been working on that a lot uh between W from Alpha and W from language we kind of have this great **2:03:42** · stack of technology that when you can represent things in the world in some kind of formalized computational way we can sort of reliably compute what will happen and what we're seeing is that llms and so on provide a great sort of linguistic interface but they're not going to be able to do that hard in the end computationally irreducible computation and so the the thing is to feed that **2:04:07** · through to our computation stack computational knowledge stack to get those things reliably done and the right now the best way to do that is to have the llm just call out to our technology stack as a tool we've done a lot of work **2:04:26** · some will be having some announcements soon actually about this that um uh relat to kind of optimally prompting the llm and and uh uh giving the llm kind of a leg up in knowing how to call our tool so to speak our tool is kind of like a neural implant for the neural net it's a **2:04:45** · it's a computational element a reliable computational element sort of grafted into the neuronet exactly how best to do that grafting that's the challenge right now that grafting is very coarse it's the neural net runs for a while then it comes up with sort of a thing that it wants to ask the tool it goes off the tool computes it comes back and then goes back into the neural net some of this work on foundations of of machine learning might suggest ways although I haven't figured out how to do it yet to make a kind of finer grained kind of **2:05:17** · version of of inserting sort of actual computation in the innards of of kind of a machine Learning System uh let's **2:05:34** · see okay a question from sakis here uh saying a stupid question from someone who has a very superficial knowledge of the subject why do the Nets always go from top to bottom and never have any transverse connections or feedback connections well yes that's a good question uh in the history of neuron Nets people often studied ones that had various kinds of feedback connections and in our brains we have various kinds of feedback connections we're well aware of that it's not just you know like the **2:06:04** · you know light comes into our eyes that hits the photo receptors and our retina it goes through a series of layers but there is feedback eventually going back to earlier layers from later ones and the reason that that's not part of the standard story of neuron Nets is because really hard to train things that have that kind of feedback it's the methods of training that are known particularly these ones that are based on back propagation and so on they you can unroll a limited a fixed number of of kind of recurrences but to do it where **2:06:36** · you say there can be any number of recurrences that's very hard to train for now maybe some of this this work we've been doing we haven't really thought about it yet um mer allow one to get some ideas about how to train kind of a Network that has arbitrary amounts of currence in it I'm not sure it's it's you're definitely uh face tof face with computational irreducibility when you're trying to do those kinds of things um yeah ww hike asks if whether **2:07:07** · forecasting is computationally irreducible and neural Nets are accelerating forecasting does this mean that forecasting wasn't as irreducible as we thought absolutely it means that there are regularities pieces of computational reducibility that we didn't know existed **2:07:23** · and that that is that's sort of always the story it's kind of like there are laws of weather forecasting that are kind of narratively explainable laws probably but we didn't happen to know them before now you know there had been sort of a tradition of weather forecasting done by humans noticing oh if you have a cold front and a warm front and a this and a that then this is what typically happens and then we had the kind of right from the ground up kind of computational solution of solving fluid dynamics with little packets of of air and water VAP and so on and sort of merging those two **2:07:54** · together and finding what are the regularities that exist that are perhaps different from the ones that people were using as human weather forecasters that's what one can kind of hope that will happen with kind of a a machine learning approach to **2:08:13** · this uh boss asks do you think machine learning will become a field of mathematics with axioms and so on that's an interesting question because in a sense well what would the theorems of machine learning be they would be things like this class of functions this class of of operations can successfully be learned by A system that looks like this um **2:08:46** · certainly one can imagine such results there are very simple-minded such results results I mentioned a few like like the result that uh a kind of a feed forward fully connected Network um Can represent let's say with with ramp with ramp activation functions can represent any piece-wise linear function that result is is Trivial basically uh and **2:09:09** · there are slightly fancier results where you have smooth activation functions that you can represent classes with smooth functions and so on those results are mathematically quite straightforward the the the question of if you con strain the size of the network characterizing what functions you can get that's not so straightforward that's not even straightforward in traditional math if you say I've got a Fier series **2:09:31** · and I'm going to keep only 10 terms of my Fier series what kinds of functions can I reproduce that's not so easy um in fact it's interesting how that interfaces with sort of the human perception of things because for example in something like jpeg one is doing three a series one's decomposing images into kind of sort of waves of different **2:09:53** · frequencies and one is using the fact that in typical images there aren't really highfrequency pieces that one cares about so one's truncating it so sort of the space of possible jpegs is a space of pixel images that only contains things that don't have too rapid a wiggle and saying well what is the space **2:10:11** · of possible jpegs that's a hard thing to answer and it's it's a hard thing to answer even in there isn't really a good a good way of talking about that other than oh I have a forer series and I chunk ated it to 10 terms so that's an interesting thing to try and characterize and sort of making the connection between what can be characterized in that sort of mathematical way and what can be characterized in the human perceptual way like it's an image that looks like it it is a picture of a cat as opposed to just a bunch of random pixels um that **2:10:42** · that connection is again very hard to make now you know you might imagine sort of an axiomatic theory that says IM imagine humans work like this imagin imagine that for example visual perception has certain feature detectors in it then with that set of feature detectors what will be the kind of uh **2:11:05** · under what circumstances can you successfully learn these particular basins of Attraction and so on can you classify assuming that those are the featur that the humans are using what you know when will a cat not look like a dog and so on those kinds of things it's been really difficult to characterize this sort of the shape of the attractor what the which Arrangements of pixels represent cats versus dogs that's been difficult in every field so in dynamical **2:11:35** · systems theory where sort of attractors and so on are are big business the one of those characterizations is the mandal BR set um that that is a characterization of well not quite a basin of Attraction but it's something very related to that the characterization of the Mandel BR set has been a piece of sort of higher mathematical art largely you know a lot **2:11:57** · of successes were were in were done in that in the in the 1980s early 1990s and so on but it's difficult it's it's not easy math and that's a very very very idealized system that um uh that looks at just this this uh function you know Z goes to z^2 plus C and so on very simple thing actually there's an even simpler version of that that I invented sometime in the 1990s that looks at just piece-wise linear versions of functions like that and which is also very hard to **2:12:30** · characterize so that's a a super simple version of Something Like characterizing A Basin of Attraction and you get the mandle BR set which is so complicated that you know it's on t-shirts everywhere type thing um so I think one can expect in these cases of sort of neural net basins of Attraction and so on or basins of attraction in these discret rul array systems and so on one can expect the same kind of complexity I mean I I um uh so that mathematics doesn't really **2:13:00** · exist yet uh could it be built is there a theory of such things quite possibly I mean quite possibly there's some kind of uh um well it's yeah it's I mean in so far as things like fractals are useful for characterizing the border of the mandal BR set um although there's more to that border than just saying it's a nested fractal pattern so similarly one can imagine things sort of a generalization **2:13:29** · of fractals being relevant to the to the boundaries of basins between the cats and dogs and so on and in image identification um it's uh but but that's pretty far away from the mathematics that is successfully done today when it comes to other uh sorts of of questions well you know the kind of systems I've been talking about like these ruler systems are much more likely to be amable to um mathematical kind of to to **2:13:59** · having mathematical things said about them I think than neurals I think that's true it's always hard to be sure of that because one of the strange things is that when you're dealing with integers and working out things about equations evolving integers for example turns out that's harder than in general than working out things involving equations involving uh Contin real numbers so the classic example of this is if you've got an equation like you know X the N plus y Nal Z the N find the values of XY X YZ **2:14:32** · and N for which that equation has integer Solutions that's really hard that's the format's Last Theorem story and in general that problem of So-Cal Diane equations equations involving the variables that that have to have values that are integers that problem is hooked into girdle's theorem it's hooked into computation irreducibility it's hooked into undid ability that problem is in general not finitely solvable however the other problem where XYZ Etc are real numbers **2:15:03** · that problem is completely solved in fact we built in in Mathematica years ago uh we built there was a a sort of theoretical solution to that problem we kind of made it practical and given any system of let's say polinomial equations and inequalities we can just grind them down and and tell you what the solutions **2:15:22** · are which you can't do in the case of integers so even though for integers if you had only a finite number of integers you just TR with cases which you obviously can't do for real numbers the real numbers have enough sort of smoothness in their structure that it turns out to be easier to sort of grab them with mathematical methods so it's not completely self-evident that the discrete case will be mathematically easier than the continuous case certainly easier to formulate what's going on than in the than in the continuous case but you know will it be **2:15:52** · possible to do things like you know derive scaling laws for for training of llms and so on I don't know I think that there will be a complicated mixture of things that have to do with the structure of these systems and things that have to do with the way the world is so for example can you explain why **2:16:10** · the nth most common word in English has frequency about one over n soal zip law no one doesn't know why you know there are various potential derivations some rather trivial ones but ultimately one doesn't know and it is a feature of maybe it's an inevitable consequence of having a language that has certain communication objectives met etc etc etc **2:16:32** · maybe that's something which one could derive from looking at a bunch of essentially machine learning like systems trying to communicate with each other and making little autoencoders that represent kind of the language communication links between them all those kinds of things but so far that's that's one could derive it only in the sense that that might be an evolutionarily optimal with respect to some Fitness function kind of result but **2:16:56** · so I think the kinds of things like these kinds of scaling laws and so on are probably a mixture of things that have to do ultimately with sort of almost biological details of how we as humans work and the kinds of things that we as humans choose to do together with the sort of raw mathematics in a sense of these underlying systems that we're studying but uh it's a very interesting question and I I have certainly been keen on the idea of an llm science or a **2:17:25** · you know a machine learning science that um uh that has some uh that where one can start sort of doing sort of physics like things where one can derive the analog of natural laws and but I think most of the time right now when we derive that like in the operation of llms it's more that we're learning features of the data like features of human language that that we're learning features of how the machine learning system works inside something I find I I **2:17:56** · I I suppose anecdotal because I don't really know the details but I you know I hear from from kids and so on oh I'm taking this machine learning class okay the the very Elementary ones they just do the three standard examples of just feeding things into a A classifier or whatever but the ones who take more theoretical ones it's like well what did you actually talk about and they usually say oh the we talked about support Vector machines which are a thing from the 1990s that was sort of a a simplified version of of kind of the way **2:18:26** · to think about U how how machine learning works it's I don't think you know it it feels right now and that was one of my points in in doing this piece of work that I was just talking about today that that it's surprisingly Barren **2:18:42** · this question of sort of what is the foundational science of for example neuronet like machine learning seems like a surprisingly baren area where you know I've encouraged many particularly physicists to get involved in this because I feel like it's a it's a sort of physics addressable problem that doesn't quite have when you when one's doing mathematics uh the way much pure **2:19:06** · mathematics is done it's kind of like let's solidly build you know axium to theorum and so on and I think that's going to be more difficult than let's kind of go a little bit more Wild West in the whole thing and say well we've got this kind of uh set up and we can deduce this from that and so on and we don't worry about the fact that you're taking in principle the limit of an infinite neural net but you can't really do that and then you take you take the limit first of the infinite neural net **2:19:32** · and then of the infinite training data or something like this if you're in the physics business you tend to just sort of say let's just assume those limits commute and we don't have to worry about the details of what arranges with what if you're in mathematics you tend to worry a lot about those kinds of things and my guess is that worrying about that makes it even more difficult to say something but I suppose the big picture here is that what I think the things I was just talking about kind of show is that one's kind of sort of one's almost **2:19:59** · drowning in computational reducibility when one's dealing with machine learning and so then the question as I was mentioning is when there is computational irreducibility there will also be the pockets of computational reducibility that will lead to things where you can say make make statements the issue is are the statements you can make aligned with things that you care about making statements about I mean it could be the case that one can say oh in **2:20:26** · this you know discret ruler Ray training system or in this neuron net there's this little weird correlation that happens when you do this kind of training and you can see these little eddes show up in the weight arrangements and things like this and you say and and **2:20:43** · then typically you would say well that's nice but you know we don't care it's just like saying there might be some particular I don't know if you're doing let's say heat transfer or something there might be some particular kind of weird mushroom shaped arrangement of how one fluid uh you know mixes with another **2:21:02** · one and it's nice and it's cool it looks like a mushroom and so on but it doesn't really matter um and so that's the question or it doesn't matter to the things we're trying to do let's say in heat transfer and so the question would be what's going to happen with that same kind of thing for the pockets of computational reducibility uh in in machine Learning Systems anyway well thanks very much for joining me and um I hope uh I hope we'll **2:21:29** · have more to talk about about these things I I might just say if you're reading the um the version of the St posted don't forget that this is kind of one of my principles in doing science you can click on any picture and you will get wol from language code that is self-contained code that will let you run and generate that picture or if doesn't work complain it should work a **2:21:53** · few of the pictures so some of those pictures they have cache data if you want to run them from the beginning there is the code to run them from the beginning but you need a slightly fancy GPU and so on to be able to actually get the results that I showed anyway thanks for joining me and uh look forward to seeing you another time another live stream another event or whatever all right bye for now for