Hey all, so I’ve been looking for ways to accelerate my loading process. I came up with an idea to break up my massive int[1024][1024][8] array into smaller chunks to be stored on the hard disk. Each chunk would be a int[256][256][8] array, so there would be a total of 16 arrays. (currently the entire int array is compressed and stored as one massive file)
My game saving is already multithreaded, it just makes a deep copy of my array on the main thread, and sends it off to another thread to be serialized. But, for loading I still do it in the main OpenGL context thread because the game needs to load the map to continue anyway. But, I had an idea; what if I used 16 different threads to load each of the [256][256][8] parts all at once, then, once all 16 are loaded whatever thread finishes last reassembles the entire map and then unlocks the OpenGL context thread?
I have to do a decent amount of refactoring just to test this, so while I play with the idea I was hoping to get some feedback on it from someone that has more multithreading experience than I do.
Basically, it would work like this:
- The OpenGL context thread flags for the map to load, sets [icode]mapLoading = true[/icode] and locks itself with a while loop [icode]while(mapLoading){Sit at loading screen, maybe animate something?}[/icode]
- 16 new threads are created, each one for a 256x256 part of my map. From what I understand of multithreading is if the threads/cores don’t exist, it’ll just wait until one opens up. So in theory I can open 16 threads regardless if the CPU is a single, dual, quad, 6 or 8 core processor. It’ll just use what it can when it can.
- Each thread loads it’s part of the map, when it’s done it sets [icode]mapLoaded[x]=true;[/icode] (where x is one of the 16 pieces).
- All threads will run an IF statement to see if all the mapLoaded booleans are true, so the last thread to run should clear the if statement and run.
- The last thread reassembles the entire 1024x1024 map from the 16 pieces then flags [icode]mapLoading = false[/icode], thus unlocking the openGL context and allowing the game to continue.
Visual illustration: