mark/reset datainputsream

Java on Mobile Devices J2ME
#1

As many handsets do not support this functionality has anyone heard about how to get round the fact that when reading in from a datainputstream (i.e. a textfile as I am doing) most phones don’t support mark/reset.

e.g. to continue to use the same inputstream by just reseting the index but as this doesnt work the bug on the early s60 hansets means that the heap space doesnt get deallocated causing out of memory errors in my program (app. closed and kernel errors have been happening).

At the moment I have been defining new inputstreams each time which is fine for s60 midp 2.0 as the space seems to clear each time.

Anybody thought of a way round this problem?

Cheers,

Goika

#2

Load the whole file into the heap?

#3

The problem is that there are many text files, some of which are quite large and cannot all be loaded into the heap at the same time. But if I am switching between the different text files all of the time instead of the heap space that the inputstream is using being freed as it is supposed to, the memory is never released. Which is causing the memory leak. Any alternatives>?

#4

O_o

The heap is huge (compared to the jar size limits) on Series 60.

How big are these text files?

#5

It is the 3650 and 7650 heaps I am having trouble with the 6600 and above do not seem to get as fragmented.
My program uses the following files:

Initialisation text files:
1x12k, 20x1k

Game text files:
30x1k and under, 4x8k(these are read in and out a lot throughout the program) - these are lookups and game text.

The game is quite heavily dependent on text, but these textfiles are causing most of the problem memory-wise - but to have them loaded into memory alongside all of the unpacked class files and their variables is proving unusable.

Is there another way to access a text file without using getresourceasstream as that is the known bug in the 3650 or instead to keep the resourcestreams open and be able to reset them without re-creating them all of the time?

#6

Thats still only 122kb of text files?

I presume some of these text files are scripts interpretted by the game?

You could compile them down to a binary format, that should reduce the heap needed to load them in by atleast a factor of 2.

Obviously you could also pack them into a single file, reduce the number of InputStreams you have to create.

#7

Thats still only 122kb of text files?
Yes, the trouble is that they have to be loaded into the heap continuously for them to be read - but on some phones the heap memory used is not released properly.

http://discussion.forum.nokia.com/forum/showthread.php?s=&threadid=59423&highlight=%2AApp+closed%2A

I presume some of these text files are scripts interpretted by the game?
mainly they are just text that we do not want to have in the heap memory all at the one time and also cannot have them hardcoded for the same reason and also for localisation purposes.

You could compile them down to a binary format, that should reduce the heap needed to load them in by atleast a factor of 2.

We have turned them into binary format externally by converting the .txt files using the same method as saving to an rms db (to a bytestream)- is this the correct method?

Obviously you could also pack them into a single file, reduce the number of InputStreams you have to create.
This would mean having to load the entire 122k file into memory everytime it would need to be read, meaning that the memory would be used up far more quickly as the inputstream cannot be reset each time.

#8

lol, I work with Graham :wink:

His advice is sound.

If you repeatedly load in resources, your game will leak memory.

Therefor logically, you must load your resources in only once

Having to load all 122kb of text into your heap at once is not so horrendous - there should be plenty of memory left over.

If there isn’t, you need to start compromising - highlight where your most expensive assets (in terms of heap) are, and reduce/remove them.
Typically this will be images

You also need to pay attention to the order and way in which you load resources - memory fragmentation is a serious issue on early Series 60’s.
Typically the best solution is to load your assets ‘largest first’.

Another optimisation that you may want to look into, is keeping the txt assets in a compressed form even when in memory.

Pure text will compress down using zlib by roughly a factor of 4.
So storing your text in a compressed zlib stream in memory will save you around 60kb of heap. (taking into account the buffers needed to decompress the zlib stream efficiently)
It will obviously be slower, but no slower than streaming it from a file in a compressed jar.
The hard bit will be writing the zlib decompressor - but im sure there are many reference implementations you can ‘borrow’.

That sounds suspiciously like coding standards, and acceptance criteria layed down by a certain publisher :wink:

#9

Thanks for your detailed advice. I am going to give the zlib solution a try, as long as the compressor code isnt going to be too much code, and therefore more heap.

What company do you and graham work for? you must be an experienced team to have both of you on board :slight_smile: