Java 4k Resources Thread

Yes, i beleive that is the case as and class with an unknown attribuite (i.e. the binary data) is not compressed by pack 200.

however it is likely that having the data as a seperate file in a jar that is pack200 compressed will still be much smaller than an embedded data in a normal jar

Compile 'n Shrink - HTTP Service

http://www.indiespot.net/app/java-four-kay

I used kzip last year, but it doesn’t support the gzip file format. Therefore here is a utility I’ve knocked up which converts zip files into gzip files. It makes various assumptions (only the first file in the zip is processed; the zip is assumed to give CRC, compressed size, and uncompressed size before the data block rather than after it). I’ve tested it with kzip and ensured that the result is correct.

Error handling is about what you’d expect from a throwaway tool (i.e. minimal).

Not yet tested on a .pack file, but I’m about to go to bed so I’ll put it up for people to play with.

import java.io.*;

// Based on http://www.gzip.org/zlib/rfc-gzip.html
// and http://www.pkware.com/documents/casestudies/APPNOTE.TXT
public class Zip2Gzip
{
	// Usage: java Zip2Gzip [file.zip [file.gzip]]
	public static void main(String[] args) throws IOException
	{
		InputStream in = args.length < 1 ? System.in : new FileInputStream(args[0]);
		in = new BufferedInputStream(in);
		OutputStream out = args.length < 2 ? System.out : new FileOutputStream(args[1]);
		out = new BufferedOutputStream(out);

		// Start by writing a gzip header.
		// Magic.
		out.write(0x1f);
		out.write(0x8b);
		// Compression method: inflate.
		out.write(0x08);
		// Flags: we include no optional extras.
		out.write(0x00);
		// Timestamp: unavailable.
		out.write(0x00);
		out.write(0x00);
		out.write(0x00);
		out.write(0x00);
		// Extra flags: none.
		out.write(0x00);
		// OS: unknown.
		out.write(0xff);

		// The next block of output is the compressed data. We need to process
		// the zip file header to find it and know how long it is.
		// local file header signature     4 bytes  (0x04034b50)
		// version needed to extract       2 bytes
		// general purpose bit flag        2 bytes
		// compression method              2 bytes
		// last mod file time              2 bytes
		// last mod file date              2 bytes
		//   Total so far:                14 bytes
		for (int i = 0; i < 14; i++) in.read();
		// crc-32                          4 bytes
		int crc1 = in.read();
		int crc2 = in.read();
		int crc3 = in.read();
		int crc4 = in.read();
		// compressed size                 4 bytes
		int cmpSz =  (in.read() & 0xff) +
		            ((in.read() & 0xff) << 8) +
		            ((in.read() & 0xff) << 16) +
		            ((in.read() & 0xff) << 24);
		// uncompressed size               4 bytes
		int ucmpSz1 = in.read();
		int ucmpSz2 = in.read();
		int ucmpSz3 = in.read();
		int ucmpSz4 = in.read();
		// file name length                2 bytes
		int nameLen = (in.read() & 0xff) + ((in.read() & 0xff) << 8);
		// extra field length              2 bytes
		int xfLen = (in.read() & 0xff) + ((in.read() & 0xff) << 8);
		// file name (variable size)
		for (int i = 0; i < nameLen; i++) in.read();
		// extra field (variable size)
		for (int i = 0; i < xfLen; i++) in.read();

		// Data follows, so we can copy it to the output.
		byte[] buf = new byte[4096];
		while (cmpSz > 0)
		{
			int desired = cmpSz > buf.length ? buf.length : cmpSz;
			int len = in.read(buf, 0, desired);
			if (len == 0) throw new EOFException();
			out.write(buf, 0, len);
			cmpSz -= len;
		}

		// The output still needs the CRC32 and the uncompressed size.
		out.write(crc1);
		out.write(crc2);
		out.write(crc3);
		out.write(crc4);
		out.write(ucmpSz1);
		out.write(ucmpSz2);
		out.write(ucmpSz3);
		out.write(ucmpSz4);

		// Done. Be tidy.
		out.close();
		in.close();
	}
}

Cooooool! :smiley:

I’m using 7z’s gz to compress the .pack file. Tomorrow I will try to see how it compares to the gz part of kzip.

Handy :slight_smile: thanks!

Uber-cool! Nice one!

Good grief. Absolutely amazing. Thanks Riven. I knew that pack200 zipped up all the files in an archive together, thus running the compression algorithm over the lot, rather than each one individually and had therefore only expected a noticeable benefit for archives containing multiple files. I guess pack200 must also reduce some of the class overhead that comes with a standard java class. I had intended to target jre1.4 again this year as the only thing I wanted out of jre1.5 was the nanosecond timer, but am now sitting down with a dazed expression. Maybe leave Falcon4k as jre1.4 as it’s pretty much finished, but on the other hand, I could fit a lot of level data in that 500 extra bytes. :o

I say just ignore Java 1.4, or you could always make 2 versions but just submit the Java 1.5 version.

IIRC at a minimum it exploits redundancy in the constant pool structure.

I don’t really get the pack200 stuff…
I have try to make a batch file to optimized my jar size (it executes itself automatically with netbean after a clean&build) :


"C:\Program Files\Java\jdk1.6.0_13\bin\jar" uf dist\k2010.jar META-INF\MANIFEST.MF

java -jar C:\proguard\lib\proguard.jar @k2010.pro

"C:\Program Files\Java\jdk1.6.0_13\bin\pack200" --effort=9 -O --no-gzip dist\k2010.jar.pack dist\k2010_.jar
"C:\Program Files\7-Zip\7z" a -tgzip -mx=9 dist\k2010.jar.gz dist\k2010.jar

echo off
echo ---------------------------------------------------------------------------
echo ---------------------------------------------------------------------------
dir dist\*.*
echo ---------------------------------------------------------------------------
echo ---------------------------------------------------------------------------

I get :


6 834 k2010.jar               // netbean jar
2 851 k2010.jar.gz          // gzip pack
4 693 k2010.jar.pack      // pack
4 078 k2010_.jar            // proguard jar

So, i think it works but are my option right ? Is it the .jar.gz that should be under the 4096 ?

Funky,

Could you also provide a means of attaching resource file(s) to be included in the resultant pack. :-*

Yes. But the name should really be k2010.pack.gz

Fantastic!!! 8) Who maintains this site? Is it a permanent project?

Thanks. Me. Yes.

I bet I can improve it further, as I’m currently not even using an obfuscator to shrink the method/field names.

Thank you.

By the help of this tool I can finally create my jar file easily. I had a big problem with compression until now.

Do you store all java classes submitted there, or can we use it without giving all your source code to you? :wink:

Yeah, it was very tempting, but I resisted. The only file that is stored, is the generated .pack.gz file, so you can download it. I can probably launch a thread to remove it after a minute or so.

I just updated the service, so that it actually gives you plain-text error messages, it something would go wrong before the external processes are launched.

I submitted my Poker4K app (2006) which was exactly 4096 bytes, and it came out at 3592 bytes.

Left 4k Dead went from 4096 bytes to 4019.

Neat. :smiley: