Hi Guys
When I use 7z to pack my jar file:
“c:\Program Files\7-zip\7z” a -mx=9 -tzip 4kg.jar build/classes/g.class META-INF/MANIFEST.MF
It packs okay, but when I double click in windows I get an error: “Invalid or corrupt jarfile…”
Any ideas?
Probably you need to put the class at the top level, so as to match the manifest. Of course I don’t know what’s in your manifest, but assuming that you are going for minimal size, then it’ll just be the class name with no path.
I have
7za a -mx=9 -mfb=255 -mpass=4 -tzip frag4k.jar G.class META-INF/MANIFEST.MF
However it is also worth trying kzip with various table sizes (/bxxx) as this can come out smaller, when you’re really feeling the pinch.
kzip /y /b128 katakanainvaders4k.jar G.class d
Alan
I know I had this problem for a while myself, unfortunately I can’t remember what it was. This post still has a purpose, try out KZip instead. It gave me waaaay better compression than 7zip (which I use for my daily needs, but KZip is to much more efficient to be ignored in this contest). Refrain from the GUI version though, it crashes on my computer while the command line version works fine.
I think now this might be something other than 7z - I tried kzip and I get the same result (though, with smaller files! ;D)
My mainfest has this in it:
Main-Class: g
(with two CR’s after the first line)
my jar file has:
g.class
meta-inf\mainfest.mf
Is all of that correct?
Thanks for any help, by the way 
/EDIT - You mispelt manifest in the above post - I assume that’s just a typo…
Strictly speaking the manifest should also have a version, although it often works without. Not sure whether omitting the version is safe on all platforms and java versions.
Manifest-Version: 1.0
Main-Class: G
The corrupt jar message normally appears when the manifest cannot be found, or it’s wrong, so I’d double check it by comparing it with one generated by the IDE.
The only other help I can give is the complete source of one of my obfustication batch files (for windows). Note that I generate a jar in the IDE, obfusticate it, then unpack, then repack it.
REM Obfusticate & compress jar
REM Use Proguard 3.2 if speed matters since 3.4 optimisation adversely effects application speed
REM Obfusticate the last compiled jar
C:\j2sdk1.4.2_10\bin\java -jar C:\User\Java\proguard3.2\lib\proguard.jar @config_1.pro
REM C:\j2sdk1.4.2_10\bin\java -jar C:\User\Java\proguard3.4\lib\proguard.jar @config_2.pro
REM Unpack the obfusticated jar
C:\j2sdk1.4.2_10\bin\jar xf C:\User\Java\Kanainvaders4k\dist\kanainvaders4k.jar
del kanainvaders4k.jar
REM Construct hiraganainvaders4k
REM crush the png
del d
copy ..\h t
pngcrush -bit_depth 1 -brute -l 9 -z 2 -reduce -rem alla t d
del t
REM Re-jar with maximum compression
kzip /y /r /b128 hiraganainvaders4k_m.jar G.class d META-INF/MANIFEST.MF
kzip /y /b128 hiraganainvaders4k.jar G.class d
REM Construct katakanainvaders4k
REM crush the png
del d
copy ..\k t
pngcrush -bit_depth 1 -brute -l 9 -z 2 -reduce -rem alla t d
del t
REM Re-jar with maximum compression
kzip /y /r /b128 katakanainvaders4k_m.jar G.class d META-INF/MANIFEST.MF
kzip /y /b128 katakanainvaders4k.jar G.class d
pause
and another
REM Obfusticate & compress jar
REM Use Proguard 3.2 since 3.4 optimisation adversely effects application speed
C:\j2sdk1.4.2_10\bin\java -jar C:\User\Java\proguard3.2\lib\proguard.jar @config_1.pro
REM C:\j2sdk1.4.2_10\bin\java -jar C:\User\Java\proguard3.4\lib\proguard.jar @config_2.pro
C:\j2sdk1.4.2_10\bin\jar xf C:\User\Java\Frag4k\dist\frag4k.jar
del frag4k.jar
7za a -mx=9 -mfb=255 -mpass=4 -tzip frag4k.jar G.class META-INF/MANIFEST.MF
pause
I’ve never had it fail on any platform without the version.
The problem appears to be in the original post:
[quote]“c:\Program Files\7-zip\7z” a -mx=9 -tzip 4kg.jar build/classes/g.class META-INF/MANIFEST.MF
[/quote]
The zip program is going to store the “build/classes” directory in the zip file. Thus the JAR Executor won’t be able to find the class. Make sure that all your files are in the same directory, so that the structure looks like this:
> ls -r
META-INF
META-INF/MANIFEST.MF
G.class
If you’re still having problems, there’s two things I’d suggest:
-
Pack the manifest BEFORE the class file. The JVM attempts to find the manifest first in the ZIP stream, so it can cause problems if it’s out of order.
-
Watch the case of your files. Throughout your posts, you’ve changed the case of “G.class” and “META-INF/MANIFEST.MF”. The class file MUST match the same case used inside the source file, and the manifest MUST be in upper case.
If you’re still having troubles, post the output of “jar -tvf 4kg.jar”
Good catch:
C:\development\java\work\4kg>jar -tvf 4kg.jar
Error in JAR file format. zip-style comment?
I opened the jar file with Winzip and indeed, ProGuard had added a zip comment to the jar. I’ve looked through the proguard docs but can’t find out how to suppress this behaviour. Any ideas? I guess I should try JoGa and see if that fixes the problem.
oh, I got it now - the jars made with kzip don’t work with java 1.5 - using the java 1.4 sdk they run okay
[quote]In my experience, if you’re getting better compression with KZip, you’re doing something wrong.
[/quote]
Sorry but I’m not giving much for that “experience”. If you look at the documentation for KZip it explains why it gets higher compression rates than most other compression tools (it takes the time to search for the optimal directory size etc). It’s still the same compression algorithm, KZip just takes the necessary time to find something closer to the optimal parameter settings.
How well data compresses is of course related to the structure of the data, but that “well written” compiled code should compress better or equally good with 7zip as with KZip is just plain wrong and in fact quite funny. I guess you know nothing about how the LZW compression works.
Could you explain how you got this “experience” in a less psuedo-scientific way?
[quote]oh, I got it now - the jars made with kzip don’t work with java 1.5 - using the java 1.4 sdk they run okay
[/quote]
I’m using Java 1.5 SDK even though I compile for 1.4 and have used KZip successfully.
kzipped jars work just fine with 1.5.
Sorry but I’m not giving much for that “experience”. If you look at the documentation for KZip it explains why it gets higher compression rates than most other compression tools (it takes the time to search for the optimal directory size etc). It’s still the same compression algorithm, KZip just takes the necessary time to find something closer to the optimal parameter settings.
[/quote]
7Zip uses the same techniques, only automatically. I’ve tested both quite a bit, and what I’ve found is that 7Zip, used properly, always comes out ahead.
[quote]How well data compresses is of course related to the structure of the data, but that “well written” compiled code should compress better or equally good with 7zip as with KZip is just plain wrong and in fact quite funny. I guess you know nothing about how the LZW compression works.
[/quote]
No, I’m quite familiar with it. What I’ve noticed is that there’s almost always more room for more features in games made by KZip users. There’s no easy way to quantify this, so use what works best for you. But I’m willing to bet that there’s a lot more you can pack in there.
Well as I said, you make no sense and I suspect that your experience have mislead you to draw this conclusion. If you could give any argument at all in a technical manner why this would be so then I would at least consider it.
I am also convinced that there is more optimizations to be made in all of our entries, but to use some kind of “magic formula” where you use 7Zip and KZip to judge this just doesn’t ring true to me. They both use the same compression algorithm, only different parameters. And what would be “good use of 7Zip”?
[quote=“DonaldEKnuth,post:12,topic:26032”]
As I said, it’s difficult to quantify on a general basis. I can look at your code and say, “If you replace that counter with a simple mathematical calculation, you’ll save a huge amount pre-compression. Since you use math bytecodes regularly, this will also translate into excellent post-compression results.” I know that you’d be reducing the number of tokens in the LZW stream, possibly avoiding a dictionary reset. But your problem over there may be unrelated to the problem some other person is having. The very nature of the beast makes it difficult to generalize about these things.
What I can say as a generalization is that the tighter your code gets, the less it responds to general compression. This is normal as you’re reducing some of the redundancy yourself. You can adjust to this by deliberately using less efficient (but more redundant!) code pre-compression, but in the end you can’t carry more information than the Shannon limit will allow. (Considerably less, actually, as LZW dosn’t get anywhere near the theoretical limits of data compression.)
Now KZip attempts to reduce the size of the file by allowing you to mess with the compression parameters to find the best balance for your file. 7Zip does much the same thing, except it does it algorithmically rather than brute-forced. My testing has shown that the 7Zip algorithms get much better results than KZip on files that are extremely difficult to compress. (e.g. Tightly written code.) KZip’s brute force, OTOH, manages to get a little extra out of files that are easily compressable.
That being said, based on your comment about KZip’s documentation (Documentation? What documentation?), I have a feeling that you’re really referring to BJWFlate rather than KZip. In which case the results may be very different.
I took the deliberty to download your game Xero which in jar file size is 4089 bytes, I then decompressed it and compressed it with KZip instead. It now occupies 3951 bytes (I have both files here if you want them) so I guess your own game is an example of this non-tight coding eh?
Do you know of any game (now that your own does not) which actually compresses better with 7Zip than KZip? It would be interesting to know, so please give me an URL to its jar-file. Also, when I said KZip documentation i meant the information available on its web page.
I’ll pull up the tests I’ve done when I get a chance. That’ s VERY interesting if you’re getting better results. I think I may know why, though. From Ken’s webpage:
12,800 bytes 11/23/2005
After several years of zero changes, it looks like he’s finally updated it. My copy is probably out of date now. I wonder if the J4K contest has anything to do with it?
[quote]I’ll pull up the tests I’ve done when I get a chance.
[/quote]
That would be very interesting!
[quote]That’ s VERY interesting if you’re getting better results.
[/quote]
Glad to have raised your interest, see for yourself if you don’t believe me.
[quote]. I wonder if the J4K contest has anything to do with it? Smiley
[/quote]
Perhaps, he has gotten some well deserved attention.
I guess this ruins your whole concept which you have argued for above? ;D
And if it doesn’t, find me one game which compresses better with 7zip. (I have both the latest version of 7zip and Kzip so it’s easy to verify.)
Ruins it? No. Modifies it? Yes. If there’s new datum in the equation, then things become interesting again. Maybe I’ll get to see if I can fit Just One More Feature™ into Xero. 
The Proguard vs. JoGa part still holds, though. I’ve taken to using both of them, but the results from Proguard optimizations were pretty disappointing in comparison to JoGa. But if you use both (thanks for that Kapta!) you get the best of both worlds.
Edit: It was Kapta that ran the tests, not Markus. My memory is failing me at my old age.
[quote]Maybe I’ll get to see if I can fit Just One More Feature™ into Xero. Wink
[/quote]
He he, that would be higher vertical speed right?
[quote]The Proguard vs. JoGa part still holds, though. I’ve taken to using both of them, but the results from Proguard optimizations were pretty disappointing in comparison to JoGa. But if you use both (thanks for that Markus!) you get the best of both worlds. Smiley
[/quote]
Now that I have your attention, does it matter in which order one applies ProGuard and JoGa? In theory it could, as one might make an optimization impossible for the other but I’m inexperienced with both of them. For my game I think I applied ProGaurd first and then JoGa.
He he, that would be higher vertical speed right?
[/quote]
I already gave you that. If you haven’t tried it already Run it again if you haven’t tried it lately. I tweaked the speed to where it feels quite natural now.
[quote]Now that I have your attention, does it matter in which order one applies ProGuard and JoGa? In theory it could, as one might make an optimization impossible for the other but I’m inexperienced with both of them. For my game I think I applied ProGaurd first and then JoGa.
[/quote]
I believe that Kapta’s findings were that Proguard then JoGa produces the best results. I’ve seen pretty much the same thing. It would appear that ProGuard and JoGa do different optimizations. Proguard’s are more general and thus don’t seem to interfere with JoGa’s. But JoGa’s optimizations seem to screw with some of the stuff that Proguard would try to modify, thus making it perform worse than it could.
Have a go @ Proguard -> Joga/Jax -> ProGuard.
Though there are numerous peephole optimisations that none of the above obfuscators perform.
For those, you can either roll your own, or have a go with jode