NIO SocketChannel Timeout

Networking & Multiplayer
#1

Hi,

The Tribal Trouble matchmaking service uses NIO selectors with non-blocking SocketChannels. However, I’ve discovered that pulling the network cable from a connected client won’t result in an error on the corresponding SelectionKey on the service. I’ve tried the socket().setSoTimeout(value) call (combined with a socket().setKeepAlive()), but it only seems to work on win32, not linux. I’d expect to get notifications of timeouts, like I get notifications of proper closing of the channel from the other end.

What is the preferred way of detecting such error conditions? I’d really like not to have to implement manual ping as that requires a periodic wakeup and fiddly code.

#2

Off the top of my head…

  1. This is the subject of outstanding bug reports (c.f. bug parade - differeing behaviour on timeout; e.g. IIRC windows has oscillated back and forth between doing something and doing nothing)

  2. This is the subject of outstanding RFE’s (ditto - but requests for extra granularity on the selectors, e.g. “add a TIMEOUT event”)

  3. Bear in mind that it is common for a TCP connection to be disconnected accidentally and for no error to be thrown and for it to be reconnected again, transparently. For instance, telnet to a server, pull your network cable (don’t type anything!), wait a few minutes, plug it back in again, and you should find you’re still connected, still logged-in, etc. I’ve seen a TCP connection survive hours this way before. But…I can’t remember if that’s what the spec demands, or if it’s just that a lot of TCP stacks happen to implement it this way when they shouldn’t.

  4. Quite a few bugs in this area were fixed in each of moving from 1.4.0 to .1 and to .2 and even in the _XX releases. So, evidence suggests it’s an active area of patching by Sun at the moment.

As to how you deal with it…you ping.

In practice, you shouldn’t need to ping much, most of the time you don’t care if a connection has idled away until/unless you’re going to send it some information or ask it something anyway. If you could provide your use-case for why you need to know it’s status, I might be able to provide a more helpful answer.

Finally, note that java has historically been very poor at handling timeouts and disconnects on all forms of networking (java.io, java.nio, etc). It’s been broken in many different ways across most platforms most of the time, so for now it’s still an area where manual hacking is necessary.

#3

Thank you for the thorough reply. The exact reason I discovered the bug is because the matchmaking service implements one-login-per-user so for example the user “elias” can only be logged in from one TT client. I had a hard crashing session while logged in, and discovered that the service kicked me the second time I tried to log in with the same user even though it was a while since the old session crashed.

To avoid this situation (and leaking connections in general for a very long running server), how do I implement pinging? I have this really nice and simple Selector.select(), but I guess this has to be timed out regularly and the connections pinged. And I probably can’t ping all of them at the same time and thereby risking a “hiccup” when (potentially) thousands of clients are connected. What a mess.

  • elias
#4

Suggestions:

  1. Any successful login MUST be you (because they had your login details), so … kick any OLD sessions automatically whenever someone logs in.

This has the benefit of being: simple, safe, and quickly discourages people who try logging in multiple times at once in order to cheat (I’m assuming there’s good reason why you disallow multiple simultaneous sessions from one user…). And…enforces a max of one at any one time.

Kicking session is easy, of course - just keep a hash of “login” to “SelectionKey”, and grab the key and cancel it when needed, so that it will drop out of the selector. I think you probably don’t need to explicitly close the connection if you do this - but check, just in case!

  1. Keep a timeout counter per SelectionKey and do the same as above (grab the key, cancel it) if no response received in X minutes. Make sure you don’t forget to put a line in the select() loop that resets the counter each time that channel is ready for READ or WRITE and has non-zero bytes ready (some Sun JVM’s still have the bug where they return a READ with -1 bytes, undocumented feature which means some form of channel disconnect).

So…you see, you really don’t need to ping. You’re getting an implicit “ping” from the client each time they send or receive data, and asssuming “no traffic in X minutes == kill you by default”. This is what I meant about, in practice, you usually don’t actually need to ping…

#5

Well I ended up with some primitive form of ping anyway, since the hard part was really the timer - waking up from select() once in a while and locate the timed out connections. The actual pinging (sending keep alive data once in a while) was simple.

  • elias
#6

[quote]Well I ended up with some primitive form of ping anyway, since the hard part was really the timer - waking up from select() once in a while and locate the timed out connections. The actual pinging (sending keep alive data once in a while) was simple.

  • elias
    [/quote]
    I don’t quite understand - why do you need to “wake up” from a select?
#7

Well the service is entirely single-threaded, so I figured the only way to get at those timed out connections, the thread needs to “wake up” from select() once in a while.

  • elias
#8

The default socket timeout is something like 4 hours or something isn’t it? It’s far longer than you expect it to be that’s for certain.

Blah3’s suggestion of kicking existing sessions on subsequent login is easily the best idea.

Cas :slight_smile: