Lately, I’ve been taking another look at my davebrain app, which is a simplified neural network. In the following post, I’ll talk first about the changes I’ve made to it since my last post and some challenges faced when building this kind of simulation. Secondly, I’ll talk about some interesting observations regarding the network’s behavior.
Building a Better Brain Part 1: Speed
One of the challenges in building a randomly-generated neural network with a potentially limitless number of neurons is, of course, optimization. After all, if each brain ‘cycle’ has 500+ neurons, with each neuron connected to 250 other neurons, we’re gonna need to focus pretty heavily on optimization if we want any generations to pass (and thus any learning to happen). The original setup used CSS, ng-repeats, and <div>s to actually draw the neurons. This looked pretty cool, and allowed us to do some neat things with 3D CSS, but man, was it slow! Instead, I’ve replaced this with HTML5 <canvas>, which allows a much faster drawing speed.
In addition, I’ve decided to fully remove the 3d-mode. The minor interesting graphical improvement this afforded (a 3D ‘brain’) was vastly offset by the huge detriment to performance.
These two changes serve to generally improve the speed of the app such that, at maximum speed, we can actually begin to see some emergent behavior now.
The other major changes made to the network are in the form of the actual behavior of the network itself, and are, in my opinion, more interesting.
Building a Better Brain Part 2: Sensing and Learning
The original goal of this project was to create a network that could learn given very minimal ‘rules’. The network then controls an ‘organism’ that attempts to seek out a prey item (and ‘consume’ it). While we could pretty easily directly program the organism to hunt down and find the organism directly, that would negate the purpose of the neural network (and thus the entire app). Instead, we need to determine simple rules that, in the lingo of evolutionary biology, act as both positive and negative environmental pressure to our organism1 . These are as follows, including both ‘reset’ scenarios as well as sensing and learning:
- Hyperactivity (Seizure): If 90% or more of the neurons are ‘active’ (firing), then the organism’s brain has essentially experienced a lethal seizure. The organism is reset.
- Hypoactivity (Brain Death): If no neurons are firing, the organism is essentially experiencing brain death. Again, the organism is reset.
- Successful Hunt: If the organism ‘catches’ the prey (measured as the location of the prey and organism being within a certain threshold), then obviously it’s been successful. It’s reset, but a bonus is added to its ‘score’.
- Energy Usage (Starvation): Finally, each neuron that fires uses up a certain percentage of a total energy amount. This is akin to metabolic energy that is used and must be replenished before an organism dies of starvation/exhaustion. If this energy is used up, then the organism resets.
- Directional Tendency: As the organism moves, any movement towards the prey adds a point to the score. Any movement away from the prey subtracts a point. This is rule is actually the least ‘unguided’, as there should be nothing explicitly telling the organism that it needs to head towards the prey. I may remove this later.
- Range Finding: In the original design, each ‘eyespot’ of the organism housed (or rather, was the geometric center of) one distance sensor that sensed the distance to the prey item. In this updated, new design, each eyespot actually now houses two sensors, one long distance and one short distance. As with regular neurons, the output of these is either binary. In the case of the distance sensors, this binary state is a random var, with the likelihood of the sensor being on going up the closer the organism is. The addition of two extra sensors affords the organism a little more specificity in locating the prey.
- Scoreboard: The score, as well as a rolling average, is now graphed on a scoreboard so that you can more easily see the improvement (or lack of improvement!) over time.
Learning (Path Adjustment): Once a generation is complete - whether it’s successful or not - the network needs to be changed based on the score of that generation. This is done in a way that mimics biological learning: successful paths (measured as a score higher than the previous rolling average) are strengthened, while unsuccessful paths are weakened. The exact mathematical method of strengthening and weakening the paths depends on user choice:
- If the user sets the Path Change Method to Multiplicative, the strength of successful paths is multiplied by 1.2 if successful, and 0.8 if unsuccessful.
- If the user sets the Path Change Method to Additive, 0.02 is added to the path if successful, and 0.02 is subtracted if unsuccessful.
Building a Better Brain Part 3: Observations
- The first, and most prevalent observation about this simulation is that it’s slow. Even if I do everything to optimize it, remember that this is a combination of evolution and learning simulator, and it tends to make mistakes just as often as it makes correct choices. In many cases, it needs to be run for a least a few hours before it shows any significant improvement.
- In line with above, remember that this is a JavaScript app, and as with a lot of front-end stuff, will stop running when the tab it’s in is not focused. So run it on a machine you’re not using!
- Interestingly, the overall pattern of scores tends to be sinusoidal. That is, a relatively successful generation is almost always followed by a relatively unsuccessful generation. This makes sense in the context of how the network learns.
- For the (relatively simple) behavior that the organism needs to learn, a huge number of neurons doesn’t actually really improve it that much. This makes sense, as C. elegans has only 302 neurons at around an 8% connect rate (that is, each neuron is on average connected to around 8% of its fellow neurons).
1.I’ll note here that this simulation actually acts both as a learning simulator - that is, the organism tries to learn how to figure out how to achieve its goal - and as an evolution simulator, whereby the organism changes during each generation depending on the previous generation’s performance and an element of random mutation.