And another word of advice: check your assumptions. You state that the game gets slower over time, but this is basically an assumption over the cause of the problem: that it has something to do with something happening over time. Such an assumption could make you blind to other causes of the problem. You should check this, try to measure it objectively. E.g. make fps measurements every few secs and create a table showing fps over time, or something similar.
In other words: be really sure what the problem is before looking for the solution. Maybe the problem is: whenever I walk far from my starting point the game slows down, or: when I start up fraps to record the slowdown the game slows down
P.s. I’ve scanned through your github code but could not find anything obvious