[SOLVED] BufferedReader.ReadLine unexpectedly blocking

It has been a while since I have had to ask upon the experts here, but I think i must be missing something very simple in understanding in what situation ReadLine() is blocking.

I have a controller that launches external classes (other jvm instances) and communicate via std in and out.

These external classes (processes) are not part of the project and are provided without source code.

For easy of development, i have implemented an option where, instead of an external processes, threads for each “process” are created and the Main method of each “process” is called. Since the “processes” expect communication via stdin/out I have swapped out the System.in and System.out streams with custom streams that instead uses sockets to pretend to be std in/out for each thread.

My problem is that whilst the controller can and does receive input from the child process, the child process “hangs” on the ReadLine() call indefinitely. If i do not use a BufferedReader, and instead simply use InputStream.read() then there is no blocking and the message is successfully received by the process.

Note that the “threading” of the “external” processes does not have any effect… the same problem exists if the “process” is run in a seperate JVM.

Here is a cut down example to demonstrate:


import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.io.PrintStream;
import java.lang.reflect.Method;
import java.net.ServerSocket;
import java.net.Socket;
import java.util.Arrays;
import java.util.List;


public class BlockingInputStreamReaderTest {
	public static void main(String[] args) throws Exception
	{
		// use custom std streams to allow per thread stream redirection
		StandardStreamManager.stdIn=System.in;
		StandardStreamManager.stdOut=System.out;
		System.setIn(StandardStreamManager.getInputStreamRedirector());
		System.setOut(new PrintStream(StandardStreamManager.getOutputStreamRedirector()));
		
		
		final int port = 9876;
		final List<String> command = Arrays.asList("java TestPlayer".split(" "));
   		Thread thread = new Thread(new Runnable() 
   		{
				
			@Override
			public void run() {
				ServerSocket ss=null;
				try
				{
					ss = new ServerSocket(port);
					Socket socket = ss.accept();
					StandardStreamManager.register(socket.getInputStream(),socket.getOutputStream());

					Class playerClass;
					playerClass = getClass().getClassLoader().loadClass(command.get(1));
		   			String[] commands = command.subList(2, command.size()).toArray(new String[0]);
		   			Method method = playerClass.getMethod("main", new String[commands.length].getClass());
		   			Object[] args = new Object[1];
		   			args[0] = commands;
		   			method.invoke(null,args);
				} catch (Exception e)
				{
					e.printStackTrace();
				}
				finally
				{
					if (ss!=null)
					try {
						ss.close();
					} catch (IOException e) {
						// TODO Auto-generated catch block
						e.printStackTrace();
					}
				}
			}
		});
   		thread.start();
   		
   		PrintStream ps = null;
   		BufferedReader br = null;
   		int retries=0;
   		while (retries++<200)
   		{
   			try
   			{
	   			Socket socket = new Socket("127.0.0.1", port);
	   			ps = new PrintStream( socket.getOutputStream());
	   			br = new BufferedReader(new InputStreamReader(socket.getInputStream()));
	   			break;
   			} catch (IOException e)
   			{
   				try {
					Thread.sleep(10);
				} catch (InterruptedException e1) {
					// TODO Auto-generated catch block
					e1.printStackTrace();
				}
   			}
   		}
   		
   		System.out.println("CONTROLLER INPUT RECEIVED: "+br.readLine());
   		System.out.println("CONTROLLER INPUT RECEIVED: "+br.readLine());
   		
   		// write to player...
   		while(true)
   		{
   			ps.println("You can do it!");
   			Thread.sleep(500);
   		}
	}
}




import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.io.PrintStream;
import java.util.HashMap;

public class StandardStreamManager
{
	static public PrintStream stdOut;
	static public InputStream stdIn;
	
	
	
	static private InputStream getInputStream()
	{
		Streams streams = registry.get(Thread.currentThread());
		if (streams == null)
		{
			return stdIn;
		}
		else
		{
			return streams.is;
		}
	}
	
	static private OutputStream getOutputStream()
	{
		Streams streams = registry.get(Thread.currentThread());
		if (streams == null)
		{
			return stdOut;
		}
		else
		{
			return streams.os;
		}
	}
	
	static private class Streams
	{
		public Streams(InputStream inputStreamRedirectWorker,
				OutputStream outputStreamRedirectWorker) {
			this.is=inputStreamRedirectWorker;
			this.os=outputStreamRedirectWorker;
		}
		InputStream is;
		OutputStream os;
	}
	static HashMap<Thread, Streams> registry = new HashMap<>();
	
	static public void register(InputStream is, OutputStream os)
	{
		registry.put(Thread.currentThread(), new Streams(is,os));
	}

	public static InputStream getInputStreamRedirector() {

		return new InputStream() {
			
			@Override
			public int read() throws IOException {
				InputStream is = getInputStream();
				return is.read();
			}
		};
	}

	public static OutputStream getOutputStreamRedirector() {

		return new OutputStream() {
			
			@Override
			public void write(int data) throws IOException {
				getOutputStream().write(data);
			}
		};
	}
	
	
}


import java.io.BufferedReader;
import java.io.InputStream;
import java.io.InputStreamReader;

public class TestPlayer {

	public static void main(String[] args) throws Exception
	{
		Runtime.getRuntime().addShutdownHook(new Thread(new Runnable() {
			
			@Override
			public void run() {
				System.err.println("TEST PLAYER SHUTDOWN!");
			}
		}));
		
		System.err.println("TEST PLAYER START!");

		InputStream in = System.in;

		// send something to the controller
		System.out.println("12345,E\n45678,F");
		
		BufferedReader br = new BufferedReader(new InputStreamReader(in));
		while (true)
		{

			String line = br.readLine();
			System.err.println("TestPlayer INPUT RECEIVED: "+line);
		}

//		String line="";
//		while (true)
//		{
//			line+=(char) in.read();
//			if (line.endsWith("\n"))
//			{
//				System.err.print("TestPlayer INPUT RECEIVED: "+line);
//				line="";
//			}
//		}
	}

}

To test that the messages are being sent correctly, uncomment the commented section in TestPlayer and comment the code block above it using the BufferedReader.

Can someone give some suggestions on why BufferedReader would be blocking? Thanks

[icode]System.in[/icode] [icode].read()[/icode] itself is blocking, not the bufferedReader.

one way to get around it is to use a dedicated thread, as you do already - and utilize [icode]available()[/icode] :

only issue [icode]in.read()[/icode] if [icode]in.available() > 0[/icode] otherwise wait and try again.

I am not sure that is the complete problem… I am happy (want) the blocking to occur until new line character is read from the underlying inputstream used by the BufferedReader. Its just that it seems like it never detects it on the one side. The other side of the socket (see line 83-84 of BlockingInputStreamReaderTest code above) happily reads a line from a BufferedReader encapsulated InputStream.

Ok, I have investigated further by looking at the source code of the JRE and have found my problem: The InputStreamReader uses StreamDecoder internally… the implementation of which populates a buffer of either 32 bytes or 8192 bytes from the InputStream before return results back to the InputStreamReader and ultimately to the BufferedReader. It seems that in one direction the 32 byte buffer option is triggered and thus I see the results on that side. For the other side it must trigger the 8192 byte buffer and it blocks until that buffer is full. At which that side outputs the messages. (I found this out after accidentally leaving the test running for a while and coming back to see that it had worked!)

That is annoying for my purposes, but at least I now understand why it was behaving the way it did… I have no idea why it chose the smaller buffer on one side but the larger buffer on the otherside tho. Ah well… I will just have to role my own ReadLine routine.

Thanks for all that pondered on my question!

Taking a quick look at this, you instantaenously call

[quote]ps = new PrintStream( socket.getOutputStream());
br = new BufferedReader(new InputStreamReader(socket.getInputStream()));
[/quote]
When you create multiple objects trying to read from the stream, you get problems. Only ever create one printStream object for each connection (as you only need to output data one way), but as for reading, if you are reading images or bytes or strings, you need to switch between and you should only read when you are looking for input, not processing.

Thanks for spotting that issue.

That is not what caused the original problem (that side of the socket correctly worked :slight_smile: ) but could be an issue in the future

When you create multiple objects trying to read from the stream, you get problems. Only ever create one printStream object for each connection (as you only need to output data one way), but as for reading, if you are reading images or bytes or strings, you need to switch between and you should only read when you are looking for input, not processing.
[/quote]
Er… unless you’re talking about something else that I missed, one of those is an input stream and the other is an output. Only one object will be reading.

You picked and chose what you read from my post.

No HeroesGraveDev is correct, your advice is incorrect. He created a single object that reads from the InputStream and a single object that writes to the OutputStream. He is only every creating on stream wrapper per Input/Output stream.

Before you call the readLine() method, you should first check if the BufferedReader is ready.


if(buff_reader.ready()){

    String input = buff_reader.readLine();

}

Just call the ready() method before you read the input.

I said that because he statically made ONE stream for ONE purpose and no way to disconnect and change them. He is going to run into a problem with this method, esp because he only reads strings. If he wants to read images via ImageIO or something, he will have trouble.

If he wants to read objects in general, he could just use ObjectInputStream. But for that, the objects have to implement Serializable.

Thanks for the best practises advice. It always helps to get a reminder :slight_smile:

These classes listed here are purely for the purposes of testing my original issue of (in my point of view) unexpected blocking on ReadLine() call and are not reflective of the quality of the final code :).

Ultimately this will be a submission to create a King-of-the-Hill code golf question (similar to that of AlienWar ) where people will submit entries ( bots ) that will play a Tower-Defence game in a tournament to find the strongest bot.

Since submissions can be from any language, all communication will be performed via std in/out using only the lower 7bits of ASCII with integers and foating point numbers sent as text strings. Thus i will not need to be sending any binary (e.g. images etc)

Thanks for all your interest in my issue.

      BufferedReader br = new BufferedReader(new InputStreamReader(in));
      while (true)
      {

         String line = br.readLine();
         System.err.println("TestPlayer INPUT RECEIVED: "+line);
      }
      BufferedReader br = new BufferedReader(new InputStreamReader(in));
      String line;
      while ((line = in.readLine()) != null)
      {
         System.err.println("TestPlayer INPUT RECEIVED: "+line);
      }

Does switching to this style somehow fix your issue. I am unable to troubleshoot your issue really.

I have identified my problem (see earlier post is due the the underlying implementation of classes used by InputStreamReader. )

So unfortunately, your suggested change has no effect as it does not change the use of InputStreamReader.

I have accepted that I will have to roll-my-own Readline to achieve immediate response to messages, it is not difficult or risky to do so for my stated purpose and will most likely be as simple as below:


public String readLine()
{
   StringBuilder sb = new StringBuilder();
   int readByte = in.read();
   while (readByte>-1 && readByte!= '\n')
   {
      sb.append((char) readByte);
      readByte = in.read();
   }
   return sb.length()==0?null:sb.toString();
}

Your latest impl interprets empty lines exactly the same as EOF. You might want to change that :point:

Riven, it’s like you were bitten by a radioactive debugger in a comp sci lab!

heh thats what comes from just typing into a post with out really testing or thinking too long on the response. cheers for the heads up

Spiderriven spiderriven! Does whatever a spiderriven does.

Does closing your program after you sent data print any data out? Because it could of been a flush issue. But like I said I haven’t popped it into eclipse and actually had a go at it.

While i have not performed that particular test, I would imagine it would not have any effect (unless i close the socket) as the StreamDecoder is still waiting to fill its buffer before continuing. But as i have said, the problem is now understood and I have a solution.

Thanks