Rendering AutoImages across multiple displays

I have a little space shooter side scroller that I would like to have support multiple monitors/displays. I am using AutoImages created with GraphicsConfiguration.createCompatibleImage and rendering them on a Canvas using 3 BufferStrategies. However, when I stretch the window to span a second monitor, all of the AutoImages just stop. According to the java -Dsun.java2d.trace=log java thinks things are still rendering just fine, but nothing will move or redraw on the second monitor again unless I either resize the frame holding the canvas or restart the frame already spanning both monitors. On the original/primary/first monitor things keep rendering like normal. Do I need to listen for a frame resize and rerender all of the AutoImages every time with a refreshed GraphicsConfiguration? Should an AutoImage be able to correctly detect which device it’s being drawn to and render itself correctly or is it the programmer’s responsibility to watch for changes in graphics devices and notify the AutoImages appropriately? http://java.sun.com/j2se/1.3/docs/guide/2d/spec/j2d-awt.fm6.html was less than enlightening. Is there another source of information for correctly supporting multiple display devices that I’m not aware of?

I am running WindowXP SP1, java sdk 1.4.2_01, NVidia Ti4200 dual display, Pentium 4 2.8C. I really appreciate any help or insight.

-Immudium

I wondered about that same issue a while ago - I don’t have a multi monitor system though, so never got to test the issue.

The way i see it (though this is entirely hypothetical), is you will need to create an Managed Image[AutoImage] for each Device. (using the appropriate GraphicsConfigurations obtained from the GraphicsDevice’s)

Abuse,
Thanks for your thoughts. In my mind, what you said makes perfect sense, I just wish Java2D felt the same way. One of the symptoms I failed to mention before is that when the frame/canvas is straddling the two GraphicDevice’s the CPU usage increases considerably. So I thought surely the AutomImages must be losing their ability to be hardware accelerated possibly because the AutoImage has to be copied to the correct GraphicsDevice which would account for the higher CPU usage. Unfortunately, this doesn’t seem to be the case.

Anyway, as far as the animation freezing, I discovered that as soon as the frame/canvas intersects a different GraphicDevice other than the one it was created on, I have to recreate the BufferStrategy. Now whether this is an issue with the display driver or the BufferStrategy or by design I can’t say, but this is the only way I can keep the animation from freezing when a GraphicDevice context switch occurs.

As far as the higher cpu usage goes when the canvas instersects two GraphicsConfigurations, again I’m stumped for an explanation. As theorized, I tried creating an AutoImage for each GraphicsConfiguration from each GraphicsDevice. Then based upon the image’s draw location, I would draw the appropriate AutoImage that corresponded to the correct GraphicsConfiguration. Doing this added a little complexity in that I now had to convert the AutoImage’s user space coordinates into device space coordinates. However, doing all of this seemed to provide no benefit.

Anyway, it seems that Java2D is doing some work for the programmer behind the scenes in that an AutoImage created on one GraphicsConfiguration WILL draw on another GraphicsConfiguration as long as the BufferStrategy is recreated as I mentioned above. Also, as noted, dividing one canvas between two GraphicsConfigurations does not seem to work well at all.

Here are a few code samples just in case. Perhaps I’m doing something horribly wrong.

Here I’m initializing each AutoImage for each GraphicsConfiguration:


static public GraphicsDevice[] graphicsDeviceList = GraphicsEnvironment.getLocalGraphicsEnvironment().getScreenDevices()

public void initGraphics(int imageType) throws OutOfMemoryError
      {
            compWidth = (int)compDimensions.getWidth();
            compHeight = (int)compDimensions.getHeight();
            
            volatileImageVector.clear();
            for (int x=0;x<graphicsDeviceList.length;x++)
            {
                  GraphicsConfiguration gConfig = graphicsDeviceList[x].getDefaultConfiguration();
            
                  switch(imageType)
                  {
                        case TickerConstants.VOLATILE:
                              try
                              {
                                    VolatileImage volatileImage = gConfig.createCompatibleVolatileImage(compWidth, compHeight);
                                    volatileImageVector.add(volatileImage);
                                    restoreVolatile(gConfig, volatileImage);
                              }
                              catch(OutOfMemoryError e)
                              {
                                    this.flush();
                                    e.printStackTrace();
                                    throw e;
                              }
                              
                              break;
                        case TickerConstants.AUTO:
                              try
                              {
                                    autoImageVector.add(gConfig.createCompatibleImage(compWidth, compWidth, Transparency.OPAQUE));
                              }
                              catch(OutOfMemoryError e)
                              {
                                    this.flush();
                                    e.printStackTrace();
                                    throw e;
                              }
                              restoreAutoImage(x);
                              break;
                  }
            }
            graphicsInitialized = true;
      }

And Here I’m rendering the AutoImages for each GraphicsConfiguration:


public void render(Graphics2D g2d)
      {
            int drawX = (int)getDrawLocX();
            int drawY = (int)getDrawLocY();
            
            if (graphicsInitialized == false)
            {
                  try
                  {
                        initGraphics(getImageType());
                  }
                  catch(OutOfMemoryError e)
                  {
                        flush();
                        e.printStackTrace();
                        if (initListener != null)
                        {
                              initListener.handleMemoryError(e);
                        }
                  }
            }
            
            switch(getImageType())
            {
                  case TickerConstants.VOLATILE:
                  for (int x=0;x<volatileImageVector.size();x++)
                  {
                        VolatileImage volatileBuffer = (VolatileImage)volatileImageVector.elementAt(x);
                        for(;;)
                           {
                              g2d.drawImage(volatileBuffer, drawX, drawY, null);
                              if (isSelected())
                              {
                                    renderHilite(g2d, drawX, drawY);
                              }
                                if(volatileBuffer.contentsLost())
                                {
                                     restoreVolatile(g2d.getDeviceConfiguration(), volatileBuffer);
                                }
                                else
                                {
                                     break;
                                }
                           }
                     }
                     break;
                     case TickerConstants.AUTO:
                     
                     for(int x=0;x<autoImageVector.size();x++)
                     {
                        BufferedImage bufferedImage = (BufferedImage)autoImageVector.elementAt(x);
                           if (graphicsDeviceList[x].getDefaultConfiguration().getBounds().contains(getVirtualBounds()))
                           {
                                 g2d.drawImage(bufferedImage, (int)getDrawLocX(), (int)getDrawLocY(), null);
                           }
                           if (isSelected())
                           {
                                 renderHilite(g2d, drawX, drawY);
                           }
                     }
                     break;
            }
      }

Well, I’ve been experimenting a little more and take back some of what I said above. I tried my code on a slower machine with a completely separate second video card rather than the P4 and dual head NVidia card I’ve been using for development. It seems that creating an AutoImage for each GraphicsConfiguration makes all the difference in the world in that scenario. I suspect it isn’t such a big deal in a dual head card because it uses shared resources. However, when the second video display is a separate, dedicated pci card, the cpu maxes out and frame rate drops to a crawl if I only use one GraphicsConfiguration. When I keep a vector of images for every GC, framerate is maintained and CPU usage is equivalent to my dual head card, which isn’t as good as only using a single display, but doesn’t max out at least. I suspect I could even save a few cycles by intelligently drawing only the portions of the image that intersect each GraphicsDevice, but I’ll have to think about whether it’s worth that added complication. Anyway, just thought I would update.

The reason for the high cpu usage for a case with two video boards is the workaround we have for a case when the destination rectangle for a drawImage operation of a managed image is spanned across the screens.

We copy a part of managed image from accelerated surface on one device to the part of the screen on this device, and the rest is copied from video memory on one device to another screen via RAM (Device1 VRAM -> System RAM -> Device2 VRAM) using a GDI call, which is extremely slow.

We didn’t think this will be a typical scenario, so we didn’t optimize it.

You might want to change your code such that you have a managed image and you make two drawimage calls for each part on each screen. We will detect that you copy this image to two devices (in separate drawImage calls) and attemt to create an accelerated copy per each device, so both your drawImage calls will be accelerated.

Hope this all makes sense.

Hi trembovetski,
I was really hoping you would have some time to respond. From your description of things, it kind of makes sense why my AutoImages start behaving like regular BufferedImages since they have to be copied to system ram. As to your work around, if I understand correctly, like Abuse, you are recommending calling createImage for each GraphicsConfiguratioon of each GraphicsDevice but also, to only draw the portion of the image that intersects the appropriate GraphicsDevice. From my initial thoughts, there seems to be a number of large hurdles to getting that to work. First of all, I’m not even sure how you would draw only a portion of an image to the Graphics2D object. Assuming that there is a way to do so via another, dynamically created accelerated image, then in that case, if your image was say 50 pixels in width you would potentially have to create and render 100 new images very quickly as it moved left to right from one GraphicsDevice to another. Also, there is the question of the images having to be aware of device coordinates instead of just their normal component coordinates. If the user moved the frame then each image lived in then each would have to be notified and their offset in device space recalcuated accordingly.

Anyway, possibly one work around that I thought of that could reduce some of the micromanagement and resource overhead quite a bit is to create a Canvas for each GraphicsDevice/Configuration instead. Then, only the canvases need to be aware of where they exist in device space and the AutoImages can continue to draw themselves in happy ignorance of the added complexity the canvases are handling. And as the user drags the parent frame from one graphicsDevice to the other, the Canvases can resize themselves appropriately to only extend as far as their GD bounds dictate.

Anyway, I’ve only begun working on this today and don’t quite have any idea if this is even feasible, but I would appreciate any thoughts you may have on this approach. Potentially, I would extend my own “DeviceCanvas” and override location and resizing information coming from the parent frame and would look something like this:


public class DeviceCanvas extends Canvas
{
      private GraphicsDevice graphicsDevice;
      
      private int numBuffers = 3;
      private Rectangle deviceBounds;
      
      public DeviceCanvas(GraphicsDevice gd)
      {
            setGraphicsDevice(gd);
      }
      
      public DeviceCanvas(int numBuffers, GraphicsDevice gd)
      {
            setGraphicsDevice(gd);
            setBufferSize(numBuffers);
            createBufferStrategy(getBufferSize());
      }
      
      public void setGraphicsDevice(GraphicsDevice gd)
      {
            graphicsDevice = gd;
            setDeviceBounds(gd.getDefaultConfiguration().getBounds());      
      }
      
      public void setDeviceBounds(Rectangle bounds)
      {
            deviceBounds = bounds;
      }
      
      public Rectangle getDeviceBounds()
      {
            return deviceBounds;
      }
      
      public int getBufferSize()
      {
            return numBuffers;
      }
      
      public void setBufferSize(int bufferSize)
      {
            numBuffers = bufferSize;
      }
      
      public void setBounds(int x, int y, int width, int height)
      {
            setBounds(new Rectangle(x, y, width, height));
      }

      public void setBounds(Rectangle r)
      {
            Rectangle newBounds = getDeviceBounds().union(r);
            super.setBounds(newBounds);
      }

      public void setLocation(int x, int y)
      {
            setBounds(x, y, getWidth(), getHeight());
      }

      public void setLocation(Point p)
      {
            setBounds(p.x, p.y, getWidth(), getHeight());
      }

      public void setSize(Dimension d)
      {
            setBounds(getX(), getY(), d.width, d.height);
      }

      public void setSize(int width, int height)
      {
            setBounds(getX(), getY(), width, height);
      }
}

Thanks again for your help.
Immudium

sorry for this not realy in sequence with everything else in the Thread.

I’ve never worked with multiple displays, so this is more curiosity than anything.
Can a single Frame span more than 1 GraphicsDevice?

if so, what does Window.getGraphicsConfiguration().getDevice() return? ???

I’d make an educated guess at it returning the primary device; but shouldn’t it return all Devices that it spans over?

ofcourse, if a single Frame can’t span more than 1 display, the question is irrelevant :wink:
but if that is the case, that prompts another question…
wtf is this Thread about lol ;D

Hey Abuse,
Yup, having a frame span two or more display devices is the entire point of this thread. When the frame in question resides entirely in the first display or entirely in the second display, the sun comes out, angels sing, in short, life is good. You can drag the frame between the two GDs all day and everything is OK as long as it’s not touching The Great, Invisible, Magical, GraphicsDevice Barrier when you release the mouse button from dragging. But the second the frame simultaneously exists on 2 or more display devices at the same time, all hell breaks loose. In my case, it’s a JFrame holding a Canvas with a BufferStrategy, but I have also tried it with your balls.jar demo in which case the same bad things happen.

Also as to your question it would seem that Frame.getGraphicsConfiguration().getDevice() returns the GD containing the majority of the frame. I would guess it would be the same when calling it from Window.

ahhh, thanx for that touch of clarity in a world gone mad.

I’m so happy I don’t have more than 1 monitor :slight_smile:

And I’m so sad. You’re like the Jav2D John Carmack. Without you, I have to actually think for myself! I’ve even half considered once or twice if I should offer to send you a free monitor ;D

[quote]And I’m so sad. You’re like the Jav2D John Carmack. Without you, I have to actually think for myself! I’ve even half considered once or twice if I should offer to send you a free monitor ;D
[/quote]
haha, I wish :o :-*

This stuff likely works differently on the Mac, since the OS X core rendering engine is fairly advanced in terms of having a frame span two independent graphics devices. All sorts of magic is done with OpenGL to make it all “just work”. The Mac uses Quartz or Quartz Extreme (hardware accelerated) as a OpenGL based rendering engine for the desktop. As I understand it drawing to a window on a Mac is basically just drawing to a texture. The OS handles the case where that texture is split over multiple graphics devices. It is still likely slow in that case though, as there isn’t much you can do to get around the fact that you have to get data to the graphics memory of different devices.

It’s interesting that you should mention that as it was a Mac guy who convinced me that I should try to add multiple displays. The more I hear about the supported 2D features of Java 1.5 on the next Mac OS version and all of the accelerated love that it’s getting, the more I become a very jealous Windows user. :’(

[quote]The more I hear about the supported 2D features of Java 1.5 on the next Mac OS version and all of the accelerated love that it’s getting, the more I become a very jealous Windows user. :’(
[/quote]
Actually I know of nothing in terms of Java 1.5 on a Mac and in general the graphics operations with Java have been a tad buggy and slow.
My comments above were about the core native graphics APIs - which the Java 1.4.1 ultimately uses… but as far as I know not in a especially clever way. There is no support yet for Volatile/Managed images for instance. (VolatileImage works, but isn’t accelerated.).
Apparently some of the speed issues are related to the core OS X APIs working only with a few pixel formats - if you have images that don’t match then there is a lot of conversions happening at runtime.
That being said, I really like the Mac and OS X. The Java stuff is there and well integrated with the OS… but still needs optimizing in some areas and just general catching up in others.

Sorry, I pulled a nasty shift in the topic by mentioning 1.5. Kind of going along with your comment about how dual monitors on Mac OS X should work a little better, I just sort of threw in what I had heard about Java2D on OS X; that it would be all OpenGL based and presumably have better hardware acceleration support for rotations, etc. whereas Window’s 1.5 would be somewhat more lacking. But, no doubt it is half rumor/speculation and part fitfull dream I had one night after too much coding. :stuck_out_tongue: But, like you I appreciate alot of what they’ve done with Mac OS. I wish I could do more development/experimenting with it.

Actually, I was suggesting to have only one managed image, but copy parts of it to the different screens, render it in tiles - a tile per screen.
Something like this:
if (image spans across screens)
for each screen
drawImage(manImage, dstlocationForThisScreen, scrLocationForThisScreen)
else
drawImage(manImage)

Hope you get the idea.

Basically, this is what we should’ve implemented in our code in the first place
No need to create more images. As I mentioned, we’ll notice that you’re copying from this managed image to different devices, and attempt to create accelerated copies on each of those devices.

I’ll see if i can put together a quick dirty prototype (kinda hard to do on a notebook)…

OK, I think I’m finally beginning to see where you’re going. Since I’ve never used tiles before per se for rendering an image, I’m still a little unsure whether I’ve actually got a handle on it, but looking back in the forums a little, I think I’m pointed in the right direction. In any case, I’ve implemented a trial run with how I think it should work which I’ll post below, but I’m not very happy with how I’m passing down the device coordinates from the parentframe to the individual images. I think I’ve kind of painted myself into a corner trying to add it after the fact, so I may start over with just an image, a frame and no animation and see if I can get it to work, then go about integrating back it into my existing code. Anyway, I appreciate your help on all of this.


public void render(Graphics2D g2d)
      {
            switch(getImageType())
            {
                  case TickerConstants.VOLATILE:
                        ...
break;
                           
                     case TickerConstants.AUTO:
                           if (graphicsDeviceList.length > 1)
                           {
                              int tileDrawLocX = 0;
                              Rectangle offsetBounds = getDataCarrierDeviceCoordinates();
                              for(int x=0;x<graphicsDeviceList.length;x++)
                              {
                                    Rectangle gdBounds = graphicsDeviceList[x].getDefaultConfiguration().getBounds();
                                    if (gdBounds.contains(offsetBounds))
                                    {
                                          g2d.drawImage(getAutoImage(), (int)getDrawLocX(), (int)getDrawLocY(), null);
                                          break;
                                    }
                                    else if (gdBounds.intersects(offsetBounds))
                                    {
                                          tileArea = graphicsDeviceList[x].getDefaultConfiguration().getBounds().union(getDataCarrierDeviceCoordinates());
                                          System.out.println("TileArea: X=" + tileArea.getX() + " Y="  + tileArea.getY() + " Width=" + tileArea.getWidth() + " Height=" + tileArea.getHeight());
                                          if (tileArea.width > 0)
                                          {
                                                g2d.drawImage(getAutoImage(), (int)getDrawLocX() + tileDrawLocX, (int)getDrawLocY(), tileArea.width, tileArea.height,
                                                                    tileDrawLocX, 0, tileArea.width, tileArea.height, null);
                                          }
                                          tileDrawLocX += tileArea.width;
                                    }
                              }
                           }
                        else
                        {
                              g2d.drawImage(getAutoImage(), (int)getDrawLocX(), (int)getDrawLocY(), null);      
                        }      
                     break;
            }
            
            if (isSelected())
            {
                  renderHilite(g2d, drawX, drawY);
            }
      }