Training a deep neural network using the genetic algorithm
In this article I will show the results of my small program in which I have implemented a genetic algorithm to train a deep neural network in the Unity game engine. All this is done in order to optimize the movement pattern of the individuals to achieve the best possible result for the given objectives. The goal may be a straight line movement as well as a high movement, which will allow us to see how the deep neural network behaves under given conditions and what the result will be, and how it can be improved in the future.

 


Figure 1: Screenshot from the app.

Source: own work

 

Neural network



The neural network will directly depend on the number of segments in the creature. In my case, this is nine segments, so the model of the neural network will look as follows:





Figure 2: ANN model concept.

Source: own work



The neural network consists of 1 input layer, 2 hidden layers and 1 output layer. In theory, such a neural network model should be able to cope with the tasks.

 



Genetic algorithm



The logic of the simulation is directly related to the genetic algorithm, which is why the general logic will look like this:





Figure 3: The concept of integrating a genetic algorithm into the overall application logic.

Source: own work


  This is roughly how the genetic algorithm will be implemented to the general logic of the application.



 




 

Simulations



Distance: ANN_2021_01_23_22_29_02



The simulation was run with the following parameters:




  • Population: 100

  • Best from population: 10

  • Generation evaluation time: 15

  • Target is Z: True

  • Crossbreeding: False

  • Smoother motion: False



The simulation time was 1 hour, 41 minutes, 51 seconds. The number of generations was 407. The maximum distance covered was 121.0147 units, and the creature reached this mark in 159 generations.


 

Video link 2: https://youtu.be/QtAkW6LK7Lw





Figure 4: Visualization of simulation data with number: ANN_2021_01_23_22_29_02.


Source: own work


 


 

Distance: ANN_2021_01_23_16_54_26



The simulation was run with the following parameters:




  • Population: 100

  • Best from population: 10

  • Generation evaluation time: 15

  • Target is Z: True

  • Crossbreeding: False

  • Smoother motion: True



The simulation time was 1 hour, 28 minutes, 22 seconds. The number of generations was 369. The maximum distance covered was 48.84888 units, and a creature reached this mark in 296 generations.


 

Video link 3: https://youtu.be/xM_sAa_Sxxw





Figure 5: Visualization of simulation data with number: ANN_2021_01_23_16_54_26


Source: own work


 


 

 



Distance: ANN_2021_01_24_22_45_08



The simulation was run with the following parameters:




  • Population: 100

  • Best from population: 10

  • Generation evaluation time: 15

  • Target is Z: True

  • Crossbreeding: False

  • Smoother motion: True



The simulation time was 1 hour, 21 minutes, 58 seconds. The number of generations was 326. The maximum distance covered was 42.90 units, which was reached by the creature in 289 generations.


 

Video link 4: https://youtu.be/DblmzdYgiRI





Figure 6: Visualization of simulation data with number: ANN_2021_01_24_22_45_08


Source: own work


 


 

Distance: ANN_2021_01_25_00_09_57



The simulation was run with the following parameters:




  • Population: 100

  • Best from population: 10

  • Generation evaluation time: 20

  • Target is Z: True

  • Crossbreeding: False

  • Smoother motion: True



The simulation time was 9 hours, 07 minutes, 42 seconds. The number of generations was 1642. The maximum distance covered was 47.23065 units, and a creature reached this mark in 1614 generations.


 

Video link 5: https://youtu.be/PSgpfKCQajc





Figure 7: Visualization of simulation data with number: ANN_2021_01_25_00_09_57


Source: own work


 


 

Distance: ANN_2021_01_23_15_12_04



The simulation was run with the following parameters:




  • Population: 100

  • Best from population: 10

  • Generation evaluation time: 15

  • Target is Z: True

  • Crossbreeding: True

  • Smoother motion: False



The simulation time was 1 hour, 41 minutes and 32 seconds. The number of generations was 405. The maximum distance covered was 114.4245 units, and the creature reached this mark in the 356th generation.


 

Video link 6: https://youtu.be/X8Kc5G8W10o





Figure 8: Visualization of simulation data with number: ANN_2021_01_23_15_12_04


Source: own work


 


 

Distance: ANN_2021_01_24_00_12_19



The simulation was run with the following parameters:




  • Population: 100

  • Best from population: 10

  • Generation evaluation time: 15

  • Target is Z: True

  • Crossbreeding: True

  • Smoother motion: False



The simulation time was 1 hour, 17 minutes, 46 seconds. The number of generations was 306. The maximum distance covered was 121.9741 units, which was reached by the creature in 272 generations.


 

Video link 7: https://youtu.be/jLS6nzlTb_Q





Figure 9: Visualization of simulation data with number: ANN_2021_01_24_00_12_19


Source: own work


 


 

Distance: ANN_2021_01_24_18_22_02



The simulation was run with the following parameters:




  • Population: 100

  • Best from population: 10

  • Generation evaluation time: 25

  • Target is Z: True

  • Crossbreeding: True

  • Smoother motion: False



The simulation time was 2 hours, 10 minutes, 56 seconds. The number of generations was 313. The maximum distance covered was 142.1166 units, which was reached by the creature in 242 generations.


 

Video link 8: https://youtu.be/CIKyYza_fe8





Figure 10: Visualization of simulation data with number: ANN_2021_01_24_18_22_02


Source: own work


 


 

Distance: ANN_2021_01_26_12_57_58



The simulation was run with the following parameters:




  • Population: 100

  • Best from population: 10

  • Generation evaluation time: 25

  • Target is Z: True

  • Crossbreeding: True

  • Smoother motion: False



The simulation time was 4 hours, 17 minutes, 28 seconds. The number of generations was 609. The maximum distance covered was 128.7373 units, which was reached by the creature in 353 generations.


 

Video link 9: https://youtu.be/gOfZ55Is3Nk





Figure 11: Visualization of simulation data with number: ANN_2021_01_26_12_57_58


Source: own work


 

 

 

Distance: ANN_2021_01_25_10_15_11



The simulation was run with the following parameters:




  • Population: 100

  • Best from population: 10

  • Generation evaluation time: 25

  • Target is Z: True

  • Crossbreeding: True

  • Smoother motion: True



The simulation time was 6 hours, 09 minutes, 57 seconds. The number of generations was 888. The maximum distance covered was 69.58593 units, which was reached by the creature in 838 generations.


 

Video link 10: https://youtu.be/RyCJOHqpnrI





Figure 12: Visualization of simulation data with number: ANN_2021_01_25_10_15_11


Source: own work


 


 

Height: ANN_2021_01_23_18_28_34



The simulation was run with the following parameters:



Population: 100




  • Best from population: 10

  • Generation evaluation time: 15

  • Target is Z: False

  • Crossbreeding: False

  • Smoother motion: On



The simulation time was 1 hour, 32 minutes and 26 seconds. The number of generations was 368. The maximum height reached was 14.56771 units, and a creature reached this mark in 171 generations.


 

Video link 11: https://youtu.be/giRKlhOcGyM





Figure 13: Visualization of simulation data with number: ANN_2021_01_23_18_28_34


Source: own work


 


 

Height: ANN_2021_01_24_14_58_18



The simulation was run with the following parameters:




  • Population: 100

  • Best from population: 10

  • Generation evaluation time: 15

  • Target is Z: False

  • Crossbreeding: False

  • Smoother motion: False



The simulation time was 1 hour, 17 minutes, 48 seconds. The number of generations was 310. The maximum height reached was 18.57985 units, and a creature reached this mark in 261 generations.


 

Video link 12: https://youtu.be/Xx1_zV5djUs





Figure 14: Visualization of simulation data with number: ANN_2021_01_24_14_58_18


Source: own work


 


 

Height: ANN_2021_05_27_05_33_24



The simulation was run with the following parameters:




  • Population: 100

  • Best from population: 10

  • Generation evaluation time: 15

  • Target is Z: False

  • Crossbreeding: False

  • Smoother motion: True



The simulation time was 1 hour, 28 minutes, 45 seconds. The number of generations was 353. The maximum height reached was 16.25196 units, and a creature reached this mark at 230 generations.


 

Video link 13: https://youtu.be/0FU0WhEI_P4





Figure 15: Visualization of simulation data with number: ANN_2021_05_27_05_33_24


Source: own work


 


 

Height: ANN_2021_01_24_16_16_45



The simulation was run with the following parameters:




  • Population: 100

  • Best from population: 10

  • Generation evaluation time: 15

  • Target is Z: False

  • Crossbreeding: False

  • Smoother motion: True



The simulation time was 1 hour, 58 minutes, 04 seconds. The number of generations was 470. The maximum height reached was 30.55556 units, and a creature reached this mark in 362 generations.


 

Video link 14: https://youtu.be/oPbtb4knl4c





Figure 16: Visualization of simulation data with number: ANN_2021_01_24_16_16_45


Source: own work


 


 

Height: ANN_2021_01_23_20_01_53



The simulation was run with the following parameters:




  • Population: 100

  • Best from population: 10

  • Generation evaluation time: 15

  • Target is Z: False

  • Crossbreeding: True

  • Smoother motion: True



The simulation time was 1 hour, 37 minutes, 56 seconds. The number of generations was 390. The maximum height reached was 19.79545 units, and a creature reached this mark in 88 generations.


 

Video link 15: https://youtu.be/RybjoJzSvDY





Figure 17: Visualization of simulation data with number: ANN_2021_01_23_20_01_53


Source: own work


 


 

Height: ANN_2021_05_27_07_04_55



The simulation was run with the following parameters:




  • Population: 100

  • Best from population: 10

  • Generation evaluation time: 15

  • Target is Z: False

  • Crossbreeding: True

  • Smoother motion: True



The simulation time was 1 hour, 32 minutes, 04 seconds. The number of generations was 366. The maximum height reached was 18.75084 units, and a creature reached this mark in the 265th generation.


 

Video link 16: https://youtu.be/iPNp1Ge-lAw





Figure 18: Visualization of simulation data with number: ANN_2021_05_27_07_04_55


Source: own work


 

 


 

Simulation results



Simulations were run with different parameters, such as population lifetime, simulation goal, with crossbreeding mechanics, without crossbreeding mechanics, with smoother and without smoother. The unchanged parameters were the number of creatures in the population, which was 100 creatures, and the mutation mechanics, since there is no provision to turn it off.  



Generation time



I analyzed the effect of the time allocated per generation. All 4 simulations have the same parameters except time. Crossbreeding and mutations are enabled, movement smoothing is disabled, and the goal is distance.





Figure 19: Visualization of data from four simulations.


Source: own work



The time difference was only 10 seconds, and looking at the graph of the best results, you can see that there is no clear line. At some intervals, the best results overlap in all 4 simulations.





Figure 20: Visualization of data from four simulations.


Source: own work


  On the data graph with the average values, you can observe a clearer difference. The best result was obtained in the simulation number ANN_2021_01_26_12_57_58, but at the same time in one of the intervals the results were almost identical to the simulation number ANN_2021_01_23_15_12_04. Since the time allocated for generation in these two simulations was 10 seconds, I can assume that the difference in time is not significant to affect the final simulation results. The difference that can be observed rather depends on the movement pattern that the creatures acquired during evolution. 

Distance



Simulations in which the target was the maximum distance were run: with mutation only, with mutation and crossing, with mutation and smoother, and with all parameters simultaneously.





Figure 21: Visualization of data from four simulations.


Source: own work


The best results clearly show the difference with certain parameters. The worst result was shown by the simulation only with smoother and mutations, but at the same time it showed almost stable results. The next bad simulation was with crossbreeding parameters, mutations and smoother, but in this case, we also managed to get very stable results. Next is a simulation in which there was only a mutation. The results were scattered, but at the same time managed to achieve the maximum result.  In addition, in the last simulation, the mechanics of crossbreeding and mutation were included; looking at the best results, we can observe almost stable best results, although a few times another simulation managed to get a better result. I can confidently say that thanks to the smoother, you can get a more stable result, but at the same time not the best.



Figure 22: Visualization of data from four simulations.


Source: own work


The average results show a noticeably larger difference than the best results. Simulations with crossbreeding mechanic show better results, while without crossbreeding it is not possible to get out of the local minimum.

Height



With height, we got more unambiguous results. The difference between the simulations was only in the parameters and as a result, we got the following.





Figure 23: Visualization of data from four simulations.


Source: own work



It is immediately apparent that with the crossbreeding mechanics we were able to get a significant difference. In other cases, however, smoother stabilized the results, but at the same time did not allow the populations to evolve more actively. Simulation with mutation at some intervals was the worst, but after some time managed to be the best among other simulations, not counting only with crossbreeding





Figure 24: Visualization of data from four simulations.


Source: own work


In terms of average results, the simulation with crossbreeding and mutations still shows the best result. It is followed by the simulation with mutations, crossbreeding and smoother, which managed to keep a stable and second best result. Finally yet importantly is the simulation with mutations and smoother and only with mutation. Based on the data, I can conclude that the simulation process works perfectly, and accomplishes all of its goals. Of course, in the future it will be possible to improve the conditions of the simulation, add the possibility of random events and other elements that will allow training creatures more effectively.
Category: Unity | Added by: Viktor_Surzhko (2021-07-10) | Author: Viktor Surzhko
Views: 2004 | Rating: 4.9/124
Total comments: 0
avatar