NIO: yet another lethal stupid bug

Networking & Multiplayer
#1

FileChannel.transferTo( Channel ) - this method can corrupt your data in 1.4.2, probably in 1.5.x too (can’t find any fixed bugs on this subject).

I just found a condition that’s causing Opera to get very upset which, when traced deep down, involves a file that is being sent using FileChannel.transferTo and dropping 2 bytes, consistently, in the same place. Suspiciously, this is precisely after sending the first 16 bytes.

Eventually, I got it narrowed down to the fact that NIO is reporting “96 bytes transferred” which happens to be the length of the source file (correct) but it is in fact only sending 94 of those bytes to the channel. This is an atomic operation (just one method in NIO) and it’s failing: I’m 99% sure this is nothing to do with my code - I don’t even think it’s possible to write code that takes stuff out of a channel that has already been put in. It’s very difficult to produce a “small” testcase because it only happens in NIO file-to-network, and you need 500 lines of code or so to get a full NIO network system running to demo with.

NB: the file in question is precisely 96 bytes in length; i.e. 2^6 + 2^5, which leads me to speculate this is a bug dependent upon a precise file length.

#2

Update: it’s not just transferto that’s broken, it’s apparently the SocketChannel itself: even doing a filechannel.read (96 bytes) to a buffer followed by a socketchannel.write (96 bytes) actuallly only writes 94 bytes.

So…my best guess right now is that some of the bytes trigger some kind of bug in socketchannel or in linux’s I/O system such that it removes them from the stream.

I’ve noticed that the actual bytes that go missing are all 255 == FF - commonly used for various “special” meanings within a stream, e.g. EOF.

So, in the stream we have:
FF FF FF
and in the output we have:
FF

Personally, my guess is that the java nio library is translating “FF FF” as “” :frowning:

#3

FYI, I have logged a bug with Sun, but I can’t comment on it and add the FF FF info until they reply to the initial bug report (which they blatantly aren’t going to for at least 3 months going on the last few bugs I filed).

Sob.

#4

so what happens if you send a binary file of 100 0xff’s ?

#5

…let me get back to you on that; wokring through a battery of tests at the moment, in the vain hope it’s something somewhere in my code, or the grexengine - or something I can workaround using NIO. At the moment, I’m stuck with no workaround :frowning: short of “edit the file and remove the FF’s”.

#6

I see the potential link between a data stream that ends exactly with FF FF FF FF when of course the EOF condition is also reported as -1. The reading of those last FF’s corresponding to the true EOF is interesting. But perhaps it is simply a red herring.

You mentioned you were testing on Linux, I wonder have you reproduced this on any other platforms?

You have only found this to be a problem with writing to SocketChannels is that right?

Have you confirmed that the FF data is necessary to make this happen?

Curiously enough I was using transferTo for a file copy and found it’s performance to be a bit slower than expected… I was going to just leave it as is, but this report has me a bit concerned that perhaps I should stick to the ‘old way’?

#7

I ff’d the original file (96 bytes of FF), and I get:

49 bytes of FF

IMHO this is definitely a bug.

Could be NIO (most likely IMHO), or could be linux (you never know…)

#8

I don’t have any other ones convenient to test upon right now, but will give it a try when I can. Would be good to know…

Only:
FROM filechannel
TO socketchannel

Yes. It only happens in that one file, which is a GIF which happens to have 3 consecutive FF’s.

Zeroing the first FF makes the bug go away entirely (i.e. 2 FF’s is fine, it’s having three that causes two bytes to be lost)

Note that I have no workaround for this bug!

Also note that there is no way of detecting the bug in your own code! (because in their infinite wisdom the sun staff implementing NIO didn’t provide any way to examine the state of a channel). The first time you learn of it will be when some important file has been corrupted. Not good.

I am beginning to wonder to what extent sun’s NIO staff deliberately broke NIO on windows and linux. NB: what follows is tongue-in-cheek…

Bear in mind that for the first 2 years after gold release NIO didn’t work on windows. Ultra basic stuff like being unable to have more than 64 clients (WTF!), and selectors that didn’t select (!). LIkewise, Sun’s linux implementation of NIO did ultra-stupid stuff like using the only form of nb/asynch I/O on linux that no-one ever uses because it’s so crap. Plus all the show-stopping bugs that hung around for > 1.5 years after gold release.

Oh, yeah, and the fact that they still haven’t written any documentation explaining how to use NIO.

Browsing NIO bug reports today, I noticed an alarming number with “closed, wontfix, couldnt reproduce” with multiple “I get this problem too” comments from different users over the course of a year or more. Having had a bug rejected once before by a sun engineer too lazy to investigate my bug, I know what it feels like (NB: just to be clear, I’ve had circa 20 bugs logged with sun engineers, most of whom were very helpful, some going way beyond what they needed to do in order to try and help fix the problem - I’ve only had the one particularly bad experience)

Makes you wonder, no?

#9

well you know Sun is slowly but surely dying … frankly I don’t think NIO is less bugged on Solaris, it’s looks more like lazyness / not the priority problem. Perhaps we should take a look at Classpath NIO implementation ?

#10

I don’t think the Windows problems were deliberate, rather just a desire to have more common code than was in fact practical. In particular the need to use different code on Windows 9x/ME and NT derived systems would have been resisted.

#11

That would be fair enough, and I did say “tongue in cheek”, but what possessed them not to even check that you could have more than 64 connections? (i.e. I believe they were using IOCP without working around the hardcoded limit of max number of channels.

#12

The usual struggle to get something out the door. Plus there are numerous places in the API which look like familiar standard (unix) methods, but behave in subtly (or not) different ways. It must have seemed attractive to use a familiar approach instead of grappling with the very different completion ports mechanism.

#13

Still no workaround, still no response from Sun (yay!).

Although I can’t find any specific record of having this bug in old JVM’s (searched the grex bugs db), I did find some customer reports of specific files failing to transfer properly which then suddenly started working again and no-one could find any problems with. Possibly this bug has always been in NIO, and it’s only now, with a very very small sample file, that anyone’s been able to spot the problem? (the customer bugs probably were on files that got altered, losing their triple-FF, and so became “safe”)

#14

OMG … it’s prolly the same bug I got in my file tranfert protocol >:( I gave up on trying to fix it and use an HTTP server for simple file transfert.

#15

Please, do the world a favour :wink: - generate as small a test case as you can and log it with Sun. If they get enough dupes of the same problem, they might realise how serious it is and fix it :slight_smile:

#16

too much assles for now :slight_smile: Jetty embedded works well for now, but I will perhaps after I released

#17

I made a simple stresser for my File server, and it finish by failling even with text files without 0xFFFF ins, so it’s not the same bug (prolly mine)