EmberIO

[blahblahblahh]

[quote]I found this article at:
http://www.theserverside.com/blogs/showblog.tss?id=DispellingNIOMyths

Perhaps you could take a look at the EmberIO lib (it looks like a NIO wrapper) and see if it solves any of your poblems?

-Chris
[/quote]
Thanks for that. Although at first-glance it looks fairly uninteresting. It seems mainly to just do the basic stuff that we did years ago, e.g.

Leaving ACCEPT out of the picture for the moment, EmberIO is setup so that data flows in the following manner:

* READ events trigger physical reads into ByteBuffers (which are under control of the app). When a READ gets a full object, it stuffs on the object on a the PROCESS FIFO queue. These READ events are generally triggered by an NIO Selector.
* PROCESS events get triggered when something gets put in the PROCESS queue. This is independent of the Selector.
* WRITE events get triggered when a user calls write() on a ReadWriteEndpoint. What happens here is that we stick the object to write on a WRITE FIFO queue, and then fire the WRITE event. In practice, if we're in non-blocking WRITE mode we try the write first, and if that doesn't work (or is incomplete) we then add OP_WRITE interest to the endpoint and wake up the Selector to deal with it.

…is about the same as someone doing a 3D engine first writing a helper class to choose a screenmode intelligently: this is basic code that anyone can write, and most experienced networking people probably did the first time they seriously used NIO. Arguably, this kind of stuff ought to be part of the standard libraries (although I’m not sure it should be, personally…see below).

I suspect official reasons why it’s not would include the argument “deployment is EXTREMELY sensitive to how this stuff is implemented, and unless your implemenation is tunable to the Nth degree it will be useless for many people”. Which is certainly true for many servers - and would make me very cautious about EmberIO (wonderng how long I could use it before I was forced to wade into the source and rewrite big chunks because they’d made assumptions that don’t hold in my environment…).

There’s some stuff in the article that is either interesting, scary, or reason to avoid Ember (depending upon how paranoid you are ;))
e.g.

Most people don’t seem to realize that Java sockets really love to deal with just one byte at first, and then open the flood gates immediately afterwards.
…
EmberIO was coded with full awareness of this odd quirk of sockets, and if a non-blocking read or write does not complete on the first try it tries it again immediately. Knowing what you know now, you won’t be surprised to hear that this tiny optimization boosts throughput by more than 50%.

Well, our server (unoptimized) is faster than Apache without any coding for this scenario; leaving me wondering whether this is:

[] Platform specific (quite likely)
[] VM-version specific (possible; there are precedents for this)
[*] Only a concern if your threads delay at certain points in their cycle (our different network architecture just might make this never appear)

I’ve done a lot of NIO debugging, and even have test cases which clearly demonstrate this is BS for particular deployments; this could be a good example of the “Ember makers assumptions that are not true and harmful” that I outlined above as a possibility.

…but of course, I’ve not tried the lib, and the linked article is not official, so coudl be peddling BS because of the ignorance of the author (happens a lot; sigh ).

[cknoll]

Not sure why a response to another thread was created into it’s own topic here, and only to flame it to death.

Also, is it now customary to label a particular project as ‘BS’ before even trying it out?

And due to the flamebait nature of this post, should this post be moved to ‘The Land of the Trolls’?

Just wondering if that is how it ‘works’ now.

-Chris

[endolf]

Hi
No, it was posted here as a discussion about EmberIO, and is irrelivant to the previous thread which was memory issues in NIO. Blah^3 tends to put things a little less delicatly than others :), but the points are genuine, not flame material at all. Having said that, it’s still interesting to see how others are handling the nasty intricasies of NIO.

Endolf

[blahblahblahh]

Also, is it now customary to label a particular project as ‘BS’ before even trying it out?

Well, I never said that, so perhaps you should re-read the post.

What I did say was that the claim that java always only reads 1 byte with NIO on the first read is BS. I say this because…it is BS. I’m sitting here right now looking at a server that doesn’t do that…

(29/04/04-09:53) PipelineParsingRKH: [DEBUG] Readable key: sun.nio.ch.SelectionKeyImpl@b03be0
(29/04/04-09:53) PipelineParsingRKH: [DEBUG] Created new inputBuffer of limit: 4000
(29/04/04-09:53) PipelineParsingRKH: [DEBUG] …read : 16 bytes…

(from a cold start, first request, manually poked via telnet to ensure nothing else got in first).

The only judgement I made on the project itself was that “it looks fairly uninteresting”. Although I did also point out one of the reasons given by vendors (like sun) for not providing this level of functionality built-in.

[cknoll]

I’ve not tried the lib, and the linked article is not official, so coudl be peddling BS

Right.

-Chris

[cknoll]

No, it was posted here as a discussion about EmberIO, and is irrelivant to the previous thread which was memory issues in NIO.

I don’t mind a discussion about EmberIO, but why does it start with a quote from some other thread (which you say is irrevilant to this thread?) Just was strange to me. Besides, there’s not much to discuss here, the poster didn’t even use the lib before passing judgement.

I find it more than a little disturbing the way certain ‘experts’ in the community will completely trash other people’s work without even performing ANY evaluation of the work before making, in my opinion, pretty harsh criticism. It’s as bad as making statements about the performance of Java without providing the facts to back those statements up (unless a huge sum of money is involved…) At the risk of getting off topic, is this really the attitude that fosters interest and knowledge building within the java gaming community? The motto is becomming ‘If you haven’t used it, trash it, and if they can’t prove it, evangalize it’. Pretty sad.

-Chris

[blahblahblahh]

I don’t mind a discussion about EmberIO, but why does it start with a quote from some other thread

As moderator, I have the responsibility to try and keep the forum clear and easy for people to follow; I considered your post was an excellent start to a new thread, but irrelevant to the thread where you had posted it (you cited a library that is designed for a completely different purpose, and could not have made any positive difference since it would automatically be subject to whatever bug I was facing).

The real reason the new thread starts with a quote of your post? Because YABB will allow me various things (like deleting or modifying your post), but misses out some others like being able to move your post :(. So I had to compromise :).

(which you say is irrevilant to this thread?) Just was strange to me. Besides, there’s not much to discuss here, the poster didn’t even use the lib before passing judgement.

FYI, I had read pretty much everything on the net about EmberIO (which is not much, and involves trawling the author’s blog).

If you want to follow up this OT discussion, post a new thread in off-topic setting out your rants, and modify your last post to also have the URL pointing to that thread. If you post anything here other than discussion of EmberIO, I’ll delete it. I don’t mind your criticisms, but it’ll just clog up this thread. Unless you really want to discuss your comments (as above), I can’t be bothered, since the points I made are obviously valid.