How I cut GTA Online loading times by 70%
GTA Online. Infamous for its slow loading times. Having picked up the game again to finish some of the newer heists I was shocked (/s) to discover that it still loads just as slow as the day it was released 7 years ago.
It was time. Time to get to the bottom of this.
Recon<br>First I wanted to check if someone had already solved this problem. Most of the results I found pointed towards anecdata about how the game is so sophisticated that it needs to load so long, stories on how the p2p network architecture is rubbish (not saying that it isn’t), some elaborate ways of loading into story mode and a solo session after that and a couple of mods that allowed skipping the startup R* logo video. Some more reading told me we could save a whopping 10-30 seconds with these combined!
Meanwhile on my PC…
Benchmark<br>Story mode load time: ~1m 10s<br>Online mode load time: ~6m flat<br>Startup menu disabled, time from R* logo until in-game (social club login time isn't counted).
Old but decent CPU: AMD FX-8350<br>Cheap-o SSD: KINGSTON SA400S37120G<br>We have to have RAM: 2x Kingston 8192 MB (DDR3-1337) 99U5471<br>Good-ish GPU: NVIDIA GeForce GTX 1070
I know my setup is dated but what on earth could take 6x longer to load into online mode? I couldn’t measure any difference using the story-to-online loading technique as others have found before me. Even if it did work the results would be down in the noise.
I Am (Not) Alone<br>If this poll is to be trusted then the issue is widespread enough to mildly annoy more than 80% of the player base. It’s been 7 years R*!
Looking around a bit to find who are the lucky ~20% that get sub 3 minute load times I came across a few benchmarks with high-end gaming PCs and an online mode load time of about 2 minutes. I would kill hack for a 2 minute load time! It does seem to be hardware-dependent but something doesn’t add up here…
How come their story mode still takes near a minute to load? (The M.2 one didn’t count the startup logos btw.) Also, loading story to online takes them only a minute more while I’m getting about five more. I know that their hardware specs are a lot better but surely not 5x better.
Highly accurate measurements<br>Armed with such powerful tools as the Task Manager I began to investigate what resources could be the bottleneck.
After taking a minute to load the common resources used for both story and online modes (which is near on par with high-end PCs) GTA decides to max out a single core on my machine for four minutes and do nothing else.
Disk usage? None! Network usage? There’s a bit, but it drops basically to zero after a few seconds (apart from loading the rotating info banners). GPU usage? Zero. Memory usage? Completely flat…
What, is it mining crypto or something? I smell code. Really bad code.
Single thread-bound<br>While my old AMD CPU has 8 cores and it does pack a punch, it was made in the olden days. Back when AMD’s single-thread performance was way behind Intel’s. This might not explain all of the load time differences but it should explain most of it.
What’s odd is that it’s using up just the CPU. I was expecting vast amounts of disk reads loading up resources or loads of network requests trying to negotiate a session in the p2p network. But this? This is probably a bug.
Profiling<br>Profilers are a great way of finding CPU bottlenecks. There’s only one problem - most of them rely on instrumenting the source code to get a perfect picture of what’s happening in the process. And I don’t have the source code. Nor do I need microsecond-perfect readings - I have 4 minutes’ worth of a bottleneck.
Enter stack sampling: for closed source applications there’s only one option. Dump the running process’ stack and current instruction pointer’s location to build a calling tree in set intervals. Then add them up to get statistics on what’s going on. There’s only one profiler that I know of (might be ignorant here) that can do this on Windows. And it hasn’t been updated in over 10 years. It’s Luke Stackwalker! Someone, please give this project some love :)
Normally Luke would group the same functions together but since I don’t have debugging symbols I had to eyeball nearby addresses to guess if it’s the same place. And what do we see? Not one bottleneck but two of them!
Down the rabbit hole<br>Having borrowed my friend’s completely legitimate copy of the industry-standard disassembler (no, I really can’t afford the thing… gonna learn to ghidra one of these days) I went to take GTA apart.
That doesn’t look right at all. Most high-profile games come with built-in protection against reverse engineering to keep away pirates, cheaters, and modders. Not that it has ever stopped them.
There seems to be some sort of an obfuscation/encryption at play here that has replaced most instructions with gibberish. Not to worry, we simply need to dump the game’s memory while it’s executing the part we want to look at. The instructions have to be...