Read File to String: can you do faster ?

Performance Tuning
#1 
String readFromFile(String strFile)
    {
    	File file = new File(strFile);
    	URI uri = file.toURI();
    	byte[] bytes = null;
    	try{
    		bytes = java.nio.file.Files.readAllBytes(java.nio.file.Paths.get(uri));
    	}catch(IOException e) { e.printStackTrace(); return "ERROR loading file "+strFile; }
	 
    	return new String(bytes);
    }

This is Java 7 mind you. Since its new I tried this and a couple of other stuff too.
This is the fastest.

Can you do faster ?

Note: A file read by this should be able to have “no” size restriction.

#2

I always do:


try
{
    BufferedReader br = new BufferedReader(new FileReader(file));
    StringBuffer str = new StringBuffer();
    String line = br.readLine();
    while (line != null)
    {
        str.append(line);
        str.append("\n");
        line = br.readLine();
    }

    return str.toString();
}
catch //etc.

I’ll bet you a zillion dollars my way is slower. And more complicated. :stuck_out_tongue: It’s just the way I first learned it and have been doing so for years.

#3

I don’t know if this speeds it up much but it’s one less append call so it can’t hurt can it?


str.append(line + "\n");
line = br.readLine();

I always read each line of a file containing text into an ArrayList so that I can get to each line by itself if I need to. Although with your method Eli you can probably just parse the line you need out of the String created by str, seems like more work to me though. This is what I do:


ArrayList<String> fileContents = new ArrayList<>(); // Please excuse the Java 7 code here, I'm lazy
BufferedReader br = new BufferedReader(new FileReader(file));
String line = br.readLine();
while(line != null)
{
   fileContents.add(line);
   line = br.readLine();
}

As for your method Cero, it’s probably fast but I don’t like it because it uses bytes and I have no idea how to work with them >< That’s my fault though so you can ignore my opinion on it >>

#4

well its only internally. using the method you give the path and it returns a string, using the bytes, but its still a normal string after that.

return new String(bytes);

Also: what about writing ? plaintext mostly, using FileWriter atm I think

#5

Oh… well I like it more now. I had no idea that you could just cast a byte[] to a String. It kind of makes sense though since a String is just a collection of bytes >_< Mind if I use your method?

As for writing I just use a BufferedWriter I belive. No idea if it’s better/worse then a FileWriter for writing plain-text.

EDIT: The only thing about your method of reading is, what do you do if you need to read from a text file that is inside a Jar?

#6

Yay. That would totally waste endless CPU cycles.

#7

O_O that sounds bad…

#8

StringBuilder has a growing char[] backing it.

A line of code like:
s = "abc"+def+"ghi"+jkl;
is turned into:
s = new StringBuilder("abc").append(def).append("ghi").append(jkl).toString();

So your suggestion would turn Eli’s code into:

StringBuilder str; ... str.append(new StringBuilder(line).append("\n").toString());

Which means at least two/three new objects per line of input. (the StringBuilder and the char[], which is grown by the append)

#9

Wow… that is worse. My bad.

#10

You don’t need permission for code this short and simple =P

Well I never do that. I structure everything like a C++ Game: Many Folders which hold all the content and an exe file.
I wold care about compression and or formats, I could just compress/decompress it myself, but who cares. Only if gets > 700MB =P

Oh yea using String concatenation while using StringBuilder.append is kinda silly. Call append each time.

Well I guess most people don’t write their own Map Editor which they rarely have to write stuff to file.

#11

The only problem I can see with your method Cero, is that you are assuming the platform’s default charset is the same as the charset used when writing your data file. (%29]new String(byte[]))

Explicitly specify “UTF-8” both when reading & writing your Strings, and you shouldn’t have any compatibility problems.

#12

oh yeah thanks, just new String(bytes, “UTF8”)

but when writing I do it like this:

void writeToFile(String msg, String path)
    {
    	try{
    		BufferedWriter bw = new BufferedWriter(new FileWriter(path));
    		bw.write(msg);
    		bw.close();
    	}catch(Exception e) { e.printStackTrace(); }
    }

Not sure how to do it then.
Well its not like the user has to open files my code writes… savegames, maps… all read by my code
maybe the console log

#13

Well, how big is your average file?

public static final String readFile(String file) throws IOException {
	BufferedInputStream in = new BufferedInputStream(new FileInputStream(file));
	ByteBuffer buffer = new ByteBuffer();
	byte[] buf = new byte[1024];
	int len;
	while ((len = in.read(buf)) != -1) {
		buffer.put(buf, len);
	}
	in.close();
	return new String(buffer.buffer, 0, buffer.write);
}

class ByteBuffer {

	public byte[] buffer = new byte[256];

	public int write;

	public void put(byte[] buf, int len) {
		ensure(len);
		System.arraycopy(buf, 0, buffer, write, len);
		write += len;
	}

	private void ensure(int amt) {
		int req = write + amt;
		if (buffer.length <= req) {
			byte[] temp = new byte[req * 2];
			System.arraycopy(buffer, 0, temp, 0, write);
			buffer = temp;
		}
	}

}

when reading a 249 byte file it was faster, when reading a 10,240,000 byte file it was faster, when reading a 10,240 byte file it was faster

just gonna assume it’s always faster

side-note: NIO is not any faster than IO

#14

Method I learned from Matzon:


private String readFile(String fileName) throws IOException {
    BufferedInputStream fin = new BufferedInputStream(new FileInputStream(fileName));
    ByteArrayOutputStream bout = new ByteArrayOutputStream();
    byte buffer[] = new byte[8192];
    int read = fin.read(buffer);
    while(read != -1) {
        bout.write(buffer, 0, read);
        read = fin.read(buffer);
    }
    fin.close();
    return new String(bout.toByteArray());
}

Tested it on file sizes: 387, 3KB and 8.3MB

The new way is faster by 2/3 on the biggest and smallest. Surprisingly my way is slightly faster(109511ns compared to 170413ns) on the middle size.

#15

Mines is a little faster :stuck_out_tongue: (maybe because it doesn’t have the abstractions of ByteArrayOutputStream?) but they look very similar, I didn’t know other people used this method, I just thought it up when reading this thread.

#16

this is fast. average 5 times faster, great stuff

#17

depending on the size of files, if you are reading really large files you might want to up the buffer from 1024 (to a max of 8192, after that there’s only minimal change as far as I can tell)

#18

Is the ‘\n’ also included on the resulted String?

#19

It reads the whole file as opposed to line by line, so yes

#20

counterp’s and CaptainJester’s/Matzon’s techniques will keep “Windows” newlines as “\r\n” in the resulting String, but Eli’s and Z-Man’s won’t, so there is a subtle difference. Not sure about Cero’s since I haven’t made the move to Java 7.