Recording OpenGL calls for later video rendering. Mad enough to work?

OpenGL Development JOGL Development
#1

Hi,

I’ve been using Fraps a bit recently, but it struggles sometimes and I have problems with broken audio, missing frames etc. I’d like to be able to record interaction with an application better than Fraps on my machine currently allows me to. I’d greatly appreciate any feedback on this. I hope someone finds this post worth reading… or lets me know why it isn’t. Of course I’d dearly love a simpler solution.

One possibility is to have an application specific recording feature, which could record all relevant aspects of the applications’ state so that it could reproduce the rendering. In a program which doesn’t already have these features (and doesn’t need them for anything else), it could mean quite a lot of not very reusable code…

So I wonder if it would be practical to implement a general method for recording all GL calls along with sufficient timing information, such that they could later be faithfully reproduced and synched with an audio recording. Either an alternative GL interface implementation could be created (similar to DebugGL), or something like AspectJ could be used to intercept calls. Copies would be made of all passed parameters… hopefully avoiding too much runtime reflection, which could be a killer if there were a lot of calls to be processed.

The amount of data would often be less than the image data that something like Fraps has to deal with. Applications constantly requiring a lot of throughput would be inappropriate, what with writing to disk not being quite as fast as PCIe-x16 ;). In some cases it would be very small; this depends a lot on the application. As it happens, some of my own work is ‘generative art’ type stuff, which often needs only a few draw calls with absolutely minute amounts of data - but which I am not able to record at decent resolution with Fraps. I would be mainly using this initially for an application with moderate amounts of dynamic geometry and some other data being shunted through GL.

The first main implementation problem that springs to mind would be dealing with handles (for textures, shaders, buffers etc); recognising when they are generated and later referenced. A lot of extra logic would need to be devoted to this, and this may involve quite a lot of hand-coding. This might be somewhat automatable by being able to recognise name like glGen*, glBind*, glGet*… but this wouldn’t catch everything and there are thousands of GL methods, plus CgGL and anything else I’ve missed out… so at first it would only support an application-specific subset of the spec. Automated tests may help as more of the spec is covered.

If anyone with better knowledge of OpenGL knows of a formal way that relevant properties could be recognised, that would be particularly useful. If I do decide to go ahead with this, I’ll start an LGPL project and of course contributions would be welcomed…

There are undoubtedly other potential gotchas that haven’t occurred to me (or have slipped my mind), but it seems to me that this could be reasonably achievable and also I hope useful.

Cheers,
Peter

#2

The guys of Oddlabs (Tribal Trouble) have a nice article on how they could re-run their games.

Google a bit, I’m sure you’ll find it.

#3

Thanks, I found it: http://oddlabs.com/blog/?p=20
Indeed, they mention this approach in relation to avoiding debugging hell, and it is quite similar to the basis of the undo functionality in another project I work on. I haven’t read the article thoroughly, but I’ll comment anyway.

[quote]If you’ve never done replay-capable games before, you’d be surprised how easy it is to incorporate, if you realize an important fact about most games: the output (graphics and sound) only depends on very few low-bandwidth input sources. And an entire run of a game can be replayed if all of the input sources are replayed.
[/quote]
Actually, this did occur to me… in fact, I swear I even removed a small bit near the beginning of my post associated to more application specific stuff, since I felt it was getting a bit long-winded… I thought for a moment it was definitely the answer, then slightly put myself off the idea…

They do go very much on the basis of a deterministic single threaded game-loop kind of architecture.

[quote]In general, you’d want to avoid multiple threads in the game loop, because the thread scheduling is not deterministic. Threads should only be used for music queueing, loading game resources and similar tasks that don’t change game behaviour.
[/quote]
At the moment, my (kind-of-a-bit-like-a-game) application is quite non-deterministic. Still, it may well be that this is a more tractable problem than the GL call logging, which is likely to have some major difficulties in implementation as well as certainly requiring much higher bandwidth. Things could perhaps be engineered to record the timing of the thread scheduling in relation to the rendering and input, such that it could be reconstructed in a deterministic way even if it was non-deterministic to begin with… but this seems to get into necessitating substantial application-specific implementation details (at least, that’s the basis on which that part of my old post grew and was thus culled).

Alternatively, I could handle input in a more game-like way (which would probably neaten things to an extent anyway) and keep physics on the same thread while recording… my performance would take a hit (although not too badly just now). Still, “In general, you’d want to avoid multiple threads in the game loop” doesn’t exactly seem like a future-proof strategy, but then I doubt logging all GL calls is either.

Making all processing deterministic and recording input may well be the saner approach. That said, Fraps isn’t performing so badly now I’ve quit some other stuff (like adobe reader ::)), although it’s still far from ideal.

#4

Your best bet, really, really, is to just get that computer upgraded. A lovely i7 with RAID and plenty of RAM. Mmmmmm.

Cas :slight_smile:

#5

Yeah, I’ll probably build myself a proper desktop fairly soon…