video display problem

Java 2D
#1

Hello!

I hope somebody can help me. Here’s the situation:
The platform, …: Windows XP, jvm 1.5.0, Windowed mode
We create a program, which displays JPG, MPEG-4 video. The displayed images represented by BufferedImages. We decode into it by native methods(using JNI). The display of the images occurs in the event loop when that calls the component’s PaintComponent method. The PaintComponent draw the image with drawImage. I attached a screenshot which shows the incorrect behavior of the displaying during video play. I moved my hand from up to down when i took this screenshot. It looks like the paintloop somehow painted the previous picture to the top and displayed the new image on the bottom. This behavior is very disturbing. It looks like this situation occurs more often when we play video with a higher framerate. We tried it on several computers and all of them produced these bad images. Do anybody know why this problem occurs?
We tried to synchronize the decoding to the buffer with the PaintComponent to prevent the changing of the video buffer during a single paint but that didn’t help.
And one more question: We decode to BGR and the monitor is running in 32bit mode so as far as I know when drawImage called the picture converted to ARGB to fit the displays properties. My question is who makes the conversion the java or the videocard?

Thanks in avance,

Bendeguy

#2

Hi!

At last I found the keyword for this phenomenon. It is called TEARING:

Tearing is a display artifact that occurs when images are presented to the screen without regard for the current status of the output circuitry. It occurs because pixels are sent to the screen gradually in book order rather than instantaneously. What happens is that the output DAC reads across the same area of the screen that is bring written to, so the monitor ends up showing a half-updated image. The result is a momentary frame that has half of an old frame and half of a new one, with a clean horizontal split across (the tear). Because the location of the tear varies according to timing, it usually jumps all over the place, which can be distracting.

So it looks like I need to synchronize the picture displaying with the monitor’s diplay cycle. My application must run in windowed mode. Is there a way to do this synchronizaton in windowed mode?

Greetings,

Bendeguy

#3

IIRC the only way to do vsync with Java2D is using BufferStrategy in fullscreen mode (and even then it’s not guranteed). If running windowed then I don’t think it’s possible to do in J2D.

#4

Thanks for your quick answer. Can you tell me what does IIRC acronym stand for? Do anybody know how can Windows Media Player, mplayer, … can avoid tearing in windowed mode?

Greetings,

Bendeguy

#5

IIRC = If I remember correctly.

Media player and other apps tend to use hardware overlays ( http://en.wikipedia.org/wiki/Hardware_overlay ) which I think side steps the issue. I suspect you’d have to dip into JNI to use it from Java code though.

#6

Take a look at this thread: http://www.java-gaming.org/forums/index.php?topic=14696.0

I hope it helps,
Keith

#7

Hi!

Thx for the link and the answers. I tried the pxsync but I have a problem with it. It worked for me when I turned off the double buffering of the swing(so the display of the video frames occurs exactly when drawImage called)
Our app is a windowed swing based application. If I understood correctly then swing draws everything to a backbuffer and then blit it to the screen. So I have to call the VideoSync.waitForBeginOfVerticalBlank(); at that point where everything painted to the backbuffer(not after the individual drawImages of all video stream). How can I do this?
I switched our application to fullscreen mode. I tried to turn on vsync, but I read that vsync only available if FlipBufferStrategy used. Is this correct?
How page flipping works? I mean it looks like for me it is for games where every pixel recalculated for every frame. But what about, when I have an app with swing gui and I open a dropdown menu for example. Java paints on the screen then flips the buffers and on the new buffer there won’t be any dropdown. Or if I paint a video frame on my buffer then flip to the other. Then something change on the screen but no new video frame arrives. A bufferFlip occurs again, so the video will step back in time. Is this correct or I misunderstood something?

Greetings,

Bendeguy

#8

Well, I implemented a native video display. It uses DirectDraw and Overlay Surface. It works fine with one or two stream but when I display 3 or 4 then it runs to deadlock. I made a thread stack dump:

“AWT-EventQueue-1” prio=6 tid=0x02ec5ce8 nid=0x904 waiting on condition [0x0354f
000…0x0354fce8]
at sun.java2d.loops.Blit.Blit(Native Method)
at sun.java2d.pipe.DrawImage.blitSurfaceData(Unknown Source)
at sun.java2d.pipe.DrawImage.renderImageCopy(Unknown Source)
at sun.java2d.pipe.DrawImage.copyImage(Unknown Source)
at sun.java2d.pipe.DrawImage.copyImage(Unknown Source)
at sun.java2d.SunGraphics2D.drawImage(Unknown Source)
at sun.java2d.SunGraphics2D.drawImage(Unknown Source)
at applet.online_view.ViewPort$ImagePanel.paintComponent(Unknown Source)

    - locked <0x1f3393f0> (a applet.online_view.ViewPort$ImagePanel)
    at javax.swing.JComponent.paint(Unknown Source)
    at javax.swing.JComponent.paintChildren(Unknown Source)
    - locked <0x1e8d1dc8> (a java.awt.Component$AWTTreeLock)
    at javax.swing.JComponent.paint(Unknown Source)
    at javax.swing.JComponent.paintWithOffscreenBuffer(Unknown Source)
    at javax.swing.JComponent.paintDoubleBuffered(Unknown Source)
    at javax.swing.JComponent._paintImmediately(Unknown Source)
    at javax.swing.JComponent.paintImmediately(Unknown Source)
    at javax.swing.RepaintManager.paintDirtyRegions(Unknown Source)
    .
    .
    .

I made a stack dump for this thread with Process Explorer:

ntoskrnl.exe!KiSwapContext+0x2e
ntoskrnl.exe!KiSwapThread+0x46
ntoskrnl.exe!KeWaitForSingleObject+0x1c2
ntoskrnl.exe!KiSuspendThread+0x18
ntoskrnl.exe!KiDeliverApc+0x124
ntoskrnl.exe!KiSwapThread+0x64
ntoskrnl.exe!KeWaitForSingleObject+0x1c2
ntoskrnl.exe!NtWaitForSingleObject+0x9a
ntoskrnl.exe!KiFastCallEntry+0xf8
ntdll.dll!KiFastSystemCallRet
ntdll.dll!ZwWaitForSingleObject+0xc
kernel32.dll!WaitForSingleObjectEx+0xa8
kernel32.dll!WaitForSingleObject+0x12
jvm.dll!jmm_GetLastGCStat+0x6964
jvm.dll!AsyncGetCallTrace+0x3f41
jvm.dll!AsyncGetCallTrace+0x26cd3
jvm.dll!AsyncGetCallTrace+0x26c42
awt.dll!Java_sun_awt_image_BufImgSurfaceData_initRaster+0x1a8

My native thread stack:
ntoskrnl.exe!KiSwapContext+0x2e
ntoskrnl.exe!KiSwapThread+0x46
ntoskrnl.exe!KeWaitForSingleObject+0x1c2
ntoskrnl.exe!KiSuspendThread+0x18
ntoskrnl.exe!KiDeliverApc+0x124
ntoskrnl.exe!KiSwapThread+0x64
ntoskrnl.exe!KeWaitForSingleObject+0x1c2
ntoskrnl.exe!NtWaitForSingleObject+0x9a
ntoskrnl.exe!KiFastCallEntry+0xf8
ntdll.dll!KiFastSystemCallRet
ntdll.dll!ZwWaitForSingleObject+0xc
ntdll.dll!RtlpWaitForCriticalSection+0x132
ntdll.dll!RtlEnterCriticalSection+0x46
ddraw.dll!DD_Surface_Lock+0x18
MPEGDecoder.dll!Java_applet_utils_MPEGDecoderIPP_stopBuffering+0x2501
MPEGDecoder.dll+0x5690

So it looks like somehow they stop each other. I only use an overlay surface from the native code and paint on that. I don’t use any other surfaces. When I want to paint on the overlay surface I lock it(to get the video memory pointer) and unlock it after the drawing. Do anybody can help me with this?

Thanks in advance,

Bendeguy

#9

It looks like this deadlock somehow related to the printing to the console. I can’t reproduce the deadlock if I don’t print any message to the console(I print debug information from my streams continuously. So the print calls count increasing with the stream count). It looks like printining to the console and painting to the hardware overlay causes the deadlock somehow. I think the event queue stucks in the blitting operation because the overlay painting thread holds some video card resource. So I think this problem not related to the Blit and my DirectDraw usage but related to the print and my DirectDraw usage. Is this possible? Does anybody heard someting like this?

Greetings,

Bendeguy