The drivers and other AI in GTA V are all "rule based." They make decisions based on being plugged directly into the game engine and they're essentially all-knowing and omniscient. GTA V game AI doesn't act based on visuals like you might while playing the game.
The neural network you see here is playing based on visuals, similar to how a vision-based self-driving car might drive with only cameras, or how you might drive with your own vision.
The field of self-driving cars is an extremely complicated one that multi-billion-dollar and even trillion-dollar companies haven't yet solved.
This is a 2-person operation and these sorts of models take a very long time to train and test.
This stream is meant to show the journey to a hopefully decent self-driving car in GTA V, rather than to immediately have a self-driving car agent that's amazing.
Watching an agent like this grow is often very rewarding and interesting, at least to us, and we hope you will enjoy the process with us!
We're currently training, playing, and streaming from a Puget Workstation. More information on this workstation can be found here: https://hubs.ly/H0-By8Q0
At one of the convolutional layers, we're displaying what the kernels output.
Each square of this image is the output of a single kernel in this convolutional layer.
This gives us an idea about what's going on in one layer of the network, which can help to inform us if the model has too many dead neurons, exploding gradient, and/or many other problems.
A neural network is trained based on mapping GTA video game frames to steering, acceleration, and braking actions. Those actions are all continuous (think: gamepad) as opposed to categorical (like a keypress).
The neural network drives primarily based on game visuals, similarly to how you probably drive in GTA.
The training data is generated via our own custom NPC that drives however we want it to (aggressively, carefully, law-abidingly...etc), and this makes the data that the model you see driving attempts to fit to. This is supervised machine learning, not reinforcement learning. At the moment, the model is set to drive very aggressively.
GTA V Reboot's authors are also the authors behind the Neural Networks from Scratch book. If you want to support Harrison Kinsley & Daniel Kukieła + learn how neural networks work from scratch, then check it out: nnfs.io