Pure Java PNG loader, with efficient & clear code...

Shared Code
#1

I have been looking for some PNGDecoder for a project I am working on, all was looking weird / complexe / dirty coded and hard to hunderstand, so last night and as often … I’ve decided to reinvent the well and to built my own :slight_smile:

  • it support the 4 filter type for filter method 0
  • for now only decoding of RGB24 & RGB32 are implemented, but adding GREY & PALETTE one should be a kid game (5/6 code line more for each)
  • it is based on high efficiency & low memory usage (also did not allocate tones of buffer)
  • only critical chunks are decoded but extended should be pretty easy to add

even if it is a first shot and it still need some improvment, I have tried to make the code as clear as possible, and it may be of interrest to better understand PNG

NB : some object are missing as RGBArrayImage and Log but can be easily replaced

EDIT : please follow the thread post to get the lastest version

#2

too long … file is attached to this post

#3

Can you ‘feed’ the pixels line-by-line to the application, so that you can scale massive images?

#4

Sorry, I dont understand exacly what you means but I try an answer, so first about the implementation you can inded do whatever is possible to do with PNG format as it is a full decoder and than you can control what is done and how it is done.

about streaming and as far as I have understand :

=> PNG can break main data in several IDAT chunk
=> you can decode start of the image without the end
=> PNG doesn’t allow uncompressing part of an image without uncompressing previous data (due to the zlib compression that compress the whole data once)

I dont get the point about scaling large image ?

ps: that’s funny because I’ve worked on this decoder for such work (scaling huge streamed image) but not in the manner you descibe

#5

if anybody get problem on using it due to lack of RGBArrayImage let me know, this is just an object that represent an RGB image (with or without Alpha) and can return its integer RGB or ARGB pixels array (can be easily replaced by a BufferedImage)

#6

Handy bit of code, though it’s dependence on the java.util.zip.Inflater to decompress the DEFLATE stream limits it’s use (and performance) somewhat.

#7

you are right, yup :-, I would have prefer a pure one two but… I believe some exist (for any platform too) but really not have the motivation to look fot it

#8

here is one : http://www.jcraft.com/jzlib/

I did not look to it yet, dont know how it is complexe,performing or well designed but at least it may this time enable “very very pure java PNG” decompression :slight_smile:

#9

Well, you guessed right, as that’s exactly what I meant. I’ve written such code, for fun and profit, and it worked with an interface like:


public interface StreamingImage
{
    public void initiateImage(int w, int h);
    public void scanlineAvailable(int[] rgb);
    public void reachedEnd();
}

imageSource.stream(inputStream, new StreamingImage() {
      ...
   }
);

I really need something similar for JPEG (streaming per MCU instead of scanline), but I don’t feel like writing it ATM.

#10

as PNG I never found any clear coded JPG Decoder, and unfortunatly JPG is not as simple as PNG…

may I ask what exacly are you looking for ? more details ?

#11

Memory constraints. People are uploading photos, not knowing they are 4000x3000 pixels, and to make them usable, the server has to scale it back to a reasonable amount. Currently, for JPGs, I launch a seperate JVM with enough RAM to prevent OOME. I need massive heap because those 4000x3000 images take much more RAM than the expected 36MB. I keep the JVM heap of the ‘real app’ as small as possible to keep GCs down to millis, as garbage collections on heaps of >1024M are downright destructive for a somewhat-near-realtime app.

Besides server stuff, for some applets, I’m dealing with fixed size 64MB heaps, of which I have 10MB to spare for image resizing. The heap can be set to be bigger since 6u10, but I have to support 1.4

Anyway, long story short, reducing every MCU to 1 pixel would be my goal.

#12

[quote=“Riven,post:11,topic:34967”]
ok I better understand now, I think it is pretty doable (but may give some headack…)

may be have a look to this thread you posted in it some times ago, take a decoder and try to modify it.

reducing MCU to one pixel should not even requiere to perform IDCT : as if I am not wrong and if I remember well you look for the average color wich should be the frequency 0 (continous component signal) => the first FFT entries ( some explanation here ) (EDIT : hum… this is very only theorical… really not sure of what I said)

EIDT : Wikipedia is alos pretty nice for JPG

#13

What Nate said: GPL :frowning:

It’s really interesting as a reference, but I certainly don’t want to ‘taint’ my project with such a license.

#14

I’m browsing your code, and this worries me a bit:
in.skip(size);

InputStream.skip() is like InputStream.read(byte[]) in that it might or might not read/skip the requested amount of bytes. You might be confused by the default implementation of the skip method, but things like FileInputStream and a Socket.getInputStream can/will have their own implementation that are more likely to skip less bytes.


public static void skipFully(InputStream in, long bytes) throws IOException
{
   while(bytes > 0)
   {
      long skipped = in.skip(bytes);
      if(skipped <= 0)
         throw new EOFException();
      bytes -= skipped;
   }
}
#15

yes I said it was " a first shot" made last night, feel free to correct it and post your update. it means that skip may not skip the amount requested ? EDIT : ok I see your code, nice to know

nothing related but here is the best JPG reference I can find

#16

I have update my version with the above skipFully method, If you got some others advice / bug found, let me know I will add them and post the corrected version in a later a post

#17

When I wrote an M3G Loader for j2me (a work-around for phones that have serious bugs in their native M3GLoaders), I wrote a light weight zlib inflater (necessary because the M3G format uses DEFLATE streams in a similar way to PNG.)
It was based upon the inflater code found in LodePNG.

Although the code is dire to read (and in C shudder), it was preferable to attempting to remove the OO structure (and thus bloat) from many of the existing zlib inflater implementations.
When I was looking for a candidate library to rip-off, it became apparent that the kind of people who write zlib libraries are the same kind of people who are appalling at documenting their code!

And my comments from the time… ::slight_smile:


			/**
			 * small bytecode, 'fast', currently doesn't support windowed decompression. (only the entire stream in one go)
			 * Based upon LodePNG (see /oldStuff/LodePNG.c )  
			 */
			final int MY_INFLATOR = 0;
			/**
			 * open source zlib decompressor that meets the specification set down for the j2se libs
			 * massive bytecode.
			 * Documented a little.
			 */
			final int JAZZLIB_INFLATOR = 1;
			/**
			 * open source zlib decompressor, totally undocumented, horrible code.
			 * massive bytecode.
			 */
			final int JZLIB_INFLATOR = 2;
#18

:slight_smile: funny comments,

clearly LodePNG is just horrible to read … , no candidate to write a pure deflate ?? I looked the specs a little when I read you previous post it was seeming doable, dont know if it would requier a long time

here are the spec and as always they are hard to read… seems that people writing RFC/spec and such never found the bold/underline/etc… and other words editor like buttons…

#19

arf … hard…

[quote]Symbol Code

A 10
B 0
C 110
D 111

I.e., 0 precedes 10 which precedes 11x, and 110 and 111 are lexicographically consecutive.
[/quote]
they say 0 precedes 10 ?? It does not seems to be the case , do I missed smething ?

#20

Your code is very nice and readable, thanks for sharing :slight_smile:

I saw that the MINA apache networking project uses jzlib (http://www.jcraft.com/jzlib/) for zipping and I thought they must know the best choice so that’s what I normally use too.

By the way, if you get time it would be great to see the code of RGBArrayImage

cheers 8)
Keith