Asking for advice on an issue I'm sure other Studios or developers have faced.

I have a problem. In my game, I have a very large player state. It's contains a global state for an RPG game. It contains arrays of state objects for everything like dialogue tree read counts, quests, simple ints, bools, and strings, and AI schedules.

It's large enough that simply serializing the data causes a hiccup in the game.

My first thought was that I should move this work to another thread. I cannot guarantee that the game state will not change on the main thread while the worker thread is serializing. The player is free to interact with the game world during this time. The worker thread that handles serialization will need to operate on a copy of my game state for the reasons of thread safety and data integrity.

My concern is this: the game state is complicated enough that making a deep-copy is not an easy task. As a experienced dev, I have confidence I can make a perfect deep-copy of my game state, but my concern is that I will end up making my project 'brittle' and hard to maintain for the rest of the team. If a junior dev adds a field to a game state class, but forgets to update the clone method, I now have a very hard to spot bug in my code.

So how do other studios or developers handle this catch-22? I have to believe that this is a solved problem in Game Development, but I haven't come across any literature on it.

@MagForceSeven Thank you for your input!

Yes, I did some profiling first. It's a Unity game, and I use the JSON.net library to serialize out game data. Pretty big hitch on the CPU (around 1000ms). Some of that's probably my data design. I abused the hell out of Polymorphism and the save can't be written without including type information, which means the serializer must rely on a lot of reflection.

In the end I just made a deep copy. Junior devs be dammed =) It was relatively straightforward. In places there were some circular references and dependencies on the order of object construction. If you didn't know your way around the code very well you'd likely mess it up, but that's true of most project contributions isn't it?

As a safeguard I added a preprocess step at build time that creates a “typical” save and ensures that the original data object and the copy serialize to the exact same value. That'll help insure that bugs don't creep in undetected.

Guess I was looking for some kind of over-architected super technical solution? Nice to know sometimes it still comes down to sound fundamentals.

nullObjectPtr said:
JSON.net

This is maybe the issue here. This library especially makes a real deep serialization of your objects and writing text is often more difficult than simply serialize everything to a binary stream. Also JSON is a DOM data model, this means that anything has to be constructed into an object tree first; this means allocating memory but not efficiently, storing references and as you mentioned, reflection.

All of this is slow as hell!

My last game that created huge saves was a City Building Simulation with large world maps, resources, ground data, buildings and AI that needed to be serialized to save. We wrote our own plain binary serialization of all these. Every necessary object has had it's own implementation of a Serialize/ Deserialize function. Serialize got a binary stream and was responsible for adding it's data to it, mostly just a few fields we added via BitConverter.

Our savegames mostly were a few 100 kb in size and took arround two frames to serialize to disk. We did all of this on the main thread and nobody noticed.

However, the drawback is that you have to keep track of version incompatibility but we solved this by adding an ID to every field we serialized in every object. The rule was then to keep this IDs unique per class and never reuse them. IDs that changed were ignored on read and not set again on write and anything worked fine for us