I see a lot of talk about java.nio on this forum, and I was wondering what’s so special about it. After checking around I see it has some buffers, like an IntBuffer. So is an IntBuffer faster than an array?
When it comes to sending data to OpenGL, buffers are the only reasonable way to do it in many circumstances, and they are guaranteed fast as well.
Other than that you might use them for serverside stuff for incoming client communications.
Cas 
NIO adds a whole feature: direct access to native memory.
When only the best will do, when you’ve absolutely positively got to transfer every last buffered-byte to/from native memory (as fast as you can).
Great fro graphics cards: write to AGP memory.
Great for servers: stream data from native-mem to native-mem using DMA and bypassing the CPU.
Really this is probably a “clueless newbie” question, but we mgiht as well answer it sicne you posted it here.
Native byte buffers are is just oen of the many uses of NIO.
It also allows for:
– Non blocking IO (of aprticualr intreest is non-blocking socekts)
– Ways to build copy-less protocol stacks.
– Memory mapped IO
There are probbaly some other features Im forgetting. Peruse the Javadocs. Thats the source of all Java knowledge
[quote] – Memory mapped IO
[/quote]
The utility of which is seriously limited on Windows. Useless for files which might change size or that you may wish to delete.
There is quite a lot of Java knowledge which can’t be found in the JavaDoc’s; you have to go and test it. They have also contained misinformation. For example, it took years to convince Sun to change the documentation relating to file locking methods. They seemed unable to understand that their recommended approaches could not be made to work reliably on Windows.
Thanks for the replies, now I’d just like to get an answer to my original question: ;D
Is an IntBuffer faster than an array of int?
[quote]Thanks for the replies, now I’d just like to get an answer to my original question: ;D
Is an IntBuffer faster than an array of int?
[/quote]
Yes.
No.
Your question is meaningless. You meant to ask “is X faster than Y at doing Z in environment E?” but your post got cut short 
Yeah, what he said - where do you want to use it and when?
Cas
[quote]Thanks for the replies, now I’d just like to get an answer to my original question: ;D
Is an IntBuffer faster than an array of int?
[/quote]
In general, no. Depending on the implementation of NIO on a particular patlrom, just putting numbers in and taking numbers out in Java is likely to be equal to or slower then a simple array.
But why do you care? What problema re you trying to sovle? Arrays are blindingly fast for anything other then non-sequential access and even there they arent likely to be your real bottleneck ulnes yo uare doing MASSIVE amounst of random access (ie an MPEG video codec.)
This really feels like a premature optimization thing.
Well I’m still designing the game I will be writing, so I wanted to know what my options were. If I would be having an array of 1 million integers and I’d have to go through it every time the mouse cursor moves (for example), then I’d like to know if there’s something faster than an array.
Of course I wouldn’t have to pass through the entire array, only in the worst possible case 
Let’s say that my game is being played on a 800x600 screen, and in that case the game map would be 20x10 screens. Now, this would not be a dynamic map, but it would be drawn and saved to a file, and then during gameplay it would be read from. Let’s say that the uncompressed size of this bitmap would be 2 GB, so it would be impossible to keep the entire bitmap in the memory. How do I solve this?
I was looking at memory mapped buffers provided by Java, is this what I need? I would only need a tiny portion of the bitmap at one time, let’s say 5 MB, so I could keep that in the memory, while the rest would stay (compressed) in a file.
Or another option would be to split the bitmap into several smaller files, and then do the memory management myself, meaning I would dump one image if it would be out of screen
You’re still not quite answering the question. When we’re talking performance tuning, you can only answer a question for a very specific problem.
For your map problem, for example: this has nothing to do with ints or arrays at all. Your problem is one of paging chunks in and out of RAM in a timely fashion. Therefore you ought to be looking at splitting the map up into teeny little chunks and maintaining a working set of pages according to what you need to display. The precise speed of it then depends on the precise requirements of your map (and in any case will probably be totally disk IO limited anyway, rendering nearly all minor tweakage irrelevant)
Cas
[quote]You’re still not quite answering the question. When we’re talking performance tuning, you can only answer a question for a very specific problem.
[/quote]
I know I’m sorry, I guess I don’t precisely know what I want. I find so many new options every day and I would like to check them all out before choosing the right one.
Let’s say I have a 2 GB bitmap, and I split it into 1 million pieces, where each piece is a tile. So one tile then is roughly 1 KB. If that tile is visible I paint it on the proper place; if it’s not visible then it should not be in the memory. Or perhaps if the tile is too close to the visible tiles, so that when I scroll 5 pixels to the left it will be visible, then it should be kept in the memory for fast access.
I could create 1 million files and then read them when necessary. This I can imagine how to do, but I guess it would be slow or awkward.
Or I could have a 2 GB image containing all the pictures, compress it into let’s say 200 MB, read that file into a mapped buffer, and then use only portions that are needed. This sounds neat but I’m not sure how to do it, or even if it can be done. I’m looking at Europa Universalis 2 where they did have a 2 GB image of the world and they read only parts from it when necessary. I would like to have something similar.
Because the file is so huge, taking up the entire address space of a typical 32bit desktop OS, you can’t memory map it. What you need to do is split the map up into tiles, compress each tile individually, then I expect the best way to store it on the disk is one file per tile and reference them in some kind of LRU cache which holds at most, say, 100 tiles. Furthermore I suggest that the LRU cache loads the 8 adjacent tiles for any request in the background.
You can afford to spend, say, 16MB of actual RAM holding map tiles, so make sure that about 25 tiles fit in 16MB and size them accordingly. That should allow you to scroll around freely with too many cache misses provided your scroll speed is slower than a disk load. I expect you’ll find that a 256x256 tile will be ideal
Cas
Cas, the map method of a FileChannel allow you to map only a part of the file so what’s the problem with 2GB files ?
Because it’d still be pretty pointless - what he wants to do is treat the whole 2GB file as a single bit of memory to make life easier. However - that means having a 2GB file on the harddisk which isn’t very nice, and as he’s just going to have to map bits of it anyway - the best solution is to chop it up first, then compress the bits. This will not only be easy to do, but it’ll be efficient too.
Cas
[quote]Cas, the map method of a FileChannel allow you to map only a part of the file so what’s the problem with 2GB files ?
[/quote]
In addition to Cas’s comments, there is also a problem when you want to map another part of the fail. This will fail if too much of the memory previously mapped has yet to be released. Nor is there any way to force this memory to be released even if it is collectable. Finally there seems to be a bug with Java+Windows where memory mapped regions overlap (this includes cases where your current request overlaps a previous request that has yet to be cleared although no longer reachable).
My conclusion is that, on Windows at least, memory mapping through Java has all the utility of a chocolate teapot. This is especially unfortunate given that I have successfully used memory mapping from C++ on Windows.
On a more practical note, its not going to be fun editing, saving, moving etc. with a whopping 2gb file. For the sake of your sanity alone split it up into seperate files. Equally with patching, replacing a single 800x600 image is a piece of cake, but manually splicing in a section into a 2gb file is not going to be fun. The alternative of just replacing the entire file is too hideous to even contemplate.
[quote]Because the file is so huge, taking up the entire address space of a typical 32bit desktop OS, you can’t memory map it. What you need to do is split the map up into tiles, compress each tile individually, then I expect the best way to store it on the disk is one file per tile and reference them in some kind of LRU cache which holds at most, say, 100 tiles. Furthermore I suggest that the LRU cache loads the 8 adjacent tiles for any request in the background.
[/quote]
Its worth noting that LRU caches in java are trivial.
This wil lalos be alot faster then mmeory mappign it all. There is still copying goign on reading amemory mapped file into the Java Heap space to use it.
HashMap + SoftReference == memory sensastive LRU cache.
[quote]Well I’m still designing the game I will be writing, so I wanted to know what my options were. If I would be having an array of 1 million integers and I’d have to go through it every time the mouse cursor moves (for example), then I’d like to know if there’s something faster than an array.
[/quote]
Depedns on for what?
For a search? Sure there are lots faster data structures in ANY language.
What you are really doing here is prematurely optimizing. That never results in the fastest code. Design your code for clarity and good seperation of functiosn and then you can tune what REALLY matters when you find out what that is.
And you wont find out til your first cut is written and profiled. Thats a promise.
[quote] There is still copying goign on reading amemory mapped file into the Java Heap space to use it.
[/quote]
Seriously? I had always figured that would only be done through a Direct Byte Buffer precisely to avoid the need to copy. It would give you very “direct” access to the OS memory mapping