I was intrigued when I saw a post about software pixel manipulation in java so I asked about a good way to do direct buffer writes and a few peeps posted very good solutions.
I’m making a new thread so as not to derail the original.
One particular post I like was by ClickerMonkey using MemoryImageSource.
Tried it but somehow won’t work. Also tried passing null instead of screen but no luck.
Here’s my code but all I get is a flashing black screen.
Thanks.
Your post seems a bit vague in it’s wording. This is a bit of code to sift through and I’m away from an IDE at the moment.
Could you point me towards where you’re actually trying to do pixel manipulation? Depending on what you’re looking to do your answer can vary.
If you’re looking to access the exact value of a pixel, getting the Raster object is very useful (WritableRaster if you’re looking to actually change their values). For BufferedImages, that’s really easy, just use it’s getRaster() method, and it’ll return it’s WritableRaster.
However, dealing with BufferStrategies is different. They purposefully hide their inner workings because they perform very complex operations managing VolatileImages, keeping track of each buffer, clearing unused images, and handling optimized page-flipping or blit-buffering algorithms. For those reasons, it tries to disallow you to edit values inside of it’s system. Because of this, there is really only a backdoor way of getting the BufferStrategy’s raster, and that’s by working through the Graphics2D class.
When performing any drawing operation, the Graphics2D class creates/gets a source and destination raster, and runs them through any of it’s Composite classes to blend them together via CompositeContext.compose(). If you create your own Composite and CompositeContext subclasses, you can effectively gain legitimate access to the current BufferStrategy Raster object being used. It’s hacky, but it works.
The structure I’m using below is based on the source for AlphaComposite.
Here’s an example you can use:
For the Composite subclass
import java.awt.Composite;
import java.awt.CompositeContext;
class RasterComposite extends Composite{
//static version kept to reduce needless instantiation
//Composite objects really just exist to store the blending rule and other things needed to blend
//otherwise, Composites should be immutable
public static final RasterComposite Normal = new RasterComposite();
//bit of a singleton pattern here, but can be extended for multiple blending rules
public static final RasterComposite getInstance(){
return Normal;
}
private RasterComposite(){}
//this is the inherited method from Composite, that links Composite and CompositeContext
//for our case it's not that important to consider color models or rendering hints
//but if youd like it's just a matter of passing that stuff via constructor
public CompositeContext createContext(ColorModel cm1, ColorModel cm2, RenderingHints hints){
return new CompositeContext();
}
}
For the CompositeContext subclass
import java.awt.CompositeContext;
import java.awt.image.Raster;
import java.awt.image.WritableRaster;
class RasterCompositeContext extends CompositeContext{
//this is where you should store the destination Raster
private WritableComposite theRaster;
//an inherited method which Graphics2D uses to blend colors
//i believe that by default Graphics2D uses AlphaComposite.SrcOver to render, so we can redirect all rendering to that
public void compose(Raster dstIn, Raster srcIn, WritableRaster dstOut){
theRaster = dstOut; //try this one, but if that doesn't work, try switching it for "theRaster = dstIn.createCompatibleWritableRaster();"
AlphaComposite.SrcOver.compose(dstIn,srcIn,dstOut); //this allows pixels to be rendered normally in the meantime.
}
//another inherited method. no objects to dispose of though
public void dispose(){}
//and here's how you get the raster from your new composite context
public final WritableRaster getRaster(){
return theRaster;
}
}
Viola, and that’s how you get the Raster object out of a BufferStrategy, in theory at least. But depending on what you’re trying to do, (like if you’re trying to implement additive blending while drawing) it may just be easier to create a Composite system that does everything as part of the drawing process.
Probably another, more inefficient way of getting the pixel values is to use the Robot class and get a screen capture of your screen. This is bad because if I recall this doesn’t work in fullscreen, requires access to the window’s position/size, needs additional scaling math if you’re scaling up (because scaling down causes loss of pixel info), and during the capture, your window must be undecorated.
Hopefully I’ve provided some sort of solution to your problem. 
You’re both over complicating this, if I understand the question.
Direct pixel access of BufferedImage:
BufferedImage img = new BufferedImage(100, 100, BufferedImage.TYPE_INT_ARGB);
int[] pixels = ((DataBufferInt) img.getRaster().getDataBuffer()).getData();
Changing pixels[] will automatically change img, iirc.
To pack r, g, and b into one integer to put into pixels[]:
public static int pack(int r, int g, int b) {
//clamp to range [0, 255]
if (r < 0)
r = 0;
else if (r > 255) r = 255;
if (g < 0)
g = 0;
else if (g > 255) g = 255;
if (b < 0)
b = 0;
else if (b > 255) b = 255;
//pack it together
return (255 << 24) | (r << 16) | (g << 8) | (b);
}
You might not need the clamp part if you know for a fact that r, g, and b will never be outside the range 0-255.
To unpack a value from pixels[] to r, g, and b values:
int r = (pack & 0xFF0000) >> 16;
int g = (pack & 0xFF00) >> 8;
int b = pack & 0xFF;
Probably not that well…
If you want to do software pixel shading in a real-time render loop, you need to stop and think what you’re doing if you aren’t using a hardware accelerated pixel shader. Unless you have a particular reason not to, consider using LWJGL or another graphics API that provides sufficient access to hardware shader APIs.
It’s incredibly slow on most systems, and every time you alter an image (without using an accelerated graphics method) you have to bring it into memory to do the software pixel operations then send it back to the GPU to be ‘accelerated.’
@BurntPizza I doubt it can possibly change img by altering pixels, int is a primitive type (i.e, a value type not a reference type.)
Basically ignore everything @Jeremy and @ericpunnum have said. @BurntPizza’s approach is absolutely correct.
It’s quite possible to do simple pixel manipulation in a real-time software rendering loop. You’ll obviously get an order of magnitude more out of a shader, but Java ain’t that slow! The sadly defunct PulpCore worked exactly like that.
This is just not true, at least with BufferedImage. BufferedImages are never read back from the GPU - they are always primarily stored in Java memory, and all drawing to them happens in software. Directly manipulating the pixels is often faster than using the Graphics2D on a BufferedImage. Just be aware of issues of image management - copy into another BufferedImage if you’re only going to manipulate once and use the copy to draw to the screen. Of course, if you’re running an effect every frame, you can ignore that.
Now you really don’t know what you’re talking about! 
It’s an int[] not an int. Arrays are reference types. The getData() method gives you a direct ‘pointer’ to the image memory.
It’s me again, and I’d like to defend my particular argument.
While this is true, I didn’t go into it that much because the title of this post is “Pixel manipulation with bufferstrategy.” As such, I felt that it was more important to explain how it could be done with a BufferStrategy (which is the fastest method of double buffering provided by Java2D) - instead of a BufferedImage. I’ll agree, it is very complicated, and can be insanely slow (pretty much halves my framerate in some cases) doing certain blending algorithms.
[quote]To unpack a value from pixels[] to r, g, and b values:
int r = (pack & 0xFF0000) >> 16;
int g = (pack & 0xFF00) >> 8;
int b = pack & 0xFF;
[/quote]
Pretty sure that Raster.getPixel() performs that exact same operation under the hood, so it’s not really that necessary to jump through that many hoops, when there’s already a method to do it for you.
I’m not saying that my method is the fastest (especially compared to GPU-accelerated libraries like JOGL or LWJGL), or even that well designed, but for what the OP is asking - which is accessing pixels in a BufferStrategy - I have at least provided a solution to what he is asking rather than talking about getting them from BufferedImages. And to be fair, I’ve mostly been talking about getting the Raster, rather than how exactly to get the pixels, which from the Raster you can use plenty of different approaches.
BufferStrategy is just double/triple buffering. There’s no need to confuse pixel manipulation into that. They’re separate things that don’t rely on each other at all.
http://docs.oracle.com/javase/7/docs/api/java/awt/image/BufferStrategy.html
Per pixel operations at the software level are that slow. It depends on what you mean by ‘simple’ but obviously it is much more practical with a lower resolution image buffer.
You’re right, for bufferedimages, and with a VolatileImage it is never read back either. My description of why the operation is slow is poorly put. The reason the operation is so slow on the software level is simply due to the number of iterations and due to the fact that once you begin performing software per-pixel operations on the bufferedimage the JRE gives up trying to hardware accelerate it.
edit
It seems I was reading the wrong section of the javadoc…
I didn’t say int[] was a value-type, I said int was a value type. The reason I had trouble believing that code is simply because I couldn’t imagine the bufferedimage object returning a reference to an internal data buffer like that - especially after also implementing setData.
But I still defend my argument since the documentation claims to return a DataBuffer, which doesn’t provide access to getData like its subclass DataBufferInt - I’m not sure if that type-cast is even safe.
http://www.java-gaming.org/user-generated-content/members/54159/pxbuf.jar
source:
package com.heartpirates;
import java.awt.Canvas;
import java.awt.Dimension;
import java.awt.Graphics;
import java.awt.image.BufferStrategy;
import java.awt.image.BufferedImage;
import java.awt.image.DataBufferInt;
import javax.swing.JFrame;
public class Main extends Canvas implements Runnable {
public final int WIDTH = 300;
public final int HEIGHT = (WIDTH * 9) / 16;
BufferedImage bimg = new BufferedImage(WIDTH, HEIGHT,
BufferedImage.TYPE_INT_RGB);
int[] pixels = ((DataBufferInt) bimg.getRaster().getDataBuffer()).getData();
public boolean running, paused;
public Main() {
Dimension size = new Dimension(WIDTH, HEIGHT);
this.setMaximumSize(size);
this.setSize(size);
this.setMinimumSize(size);
}
public void start() {
running = true;
paused = false;
new Thread(this).start();
}
@Override
public void run() {
while (running) {
if (!paused) {
step();
render();
swap();
}
sleep();
}
}
private void step() {
}
private void render() {
long now = System.currentTimeMillis();
int r = (int) (now / 2000) % 6;
for (int i = 0; i < pixels.length; i++) {
switch (r) {
case 0:
pixels[i] = (int) ((now % (i + 1)) & 0xbf8f8f);
break;
case 1:
pixels[i] = (int) (now * now + i * i);
break;
case 2:
pixels[i] = (int) ((now * (i + 1)) >>> 1);
break;
case 3:
pixels[i] = (int) (now + (i * i));
break;
case 4:
pixels[i] = (int) ((now & 0x0f0f0));
break;
case 5:
pixels[i] = (int) ((now * 3 + i * i * 3) & 0xffff00);
break;
}
}
}
private void swap() {
BufferStrategy bs = this.getBufferStrategy();
if (bs == null) {
if (this.getFocusCycleRootAncestor() != null)
this.createBufferStrategy(2);
return;
}
Graphics g = bs.getDrawGraphics();
g.drawImage(bimg, 0, 0, this.getWidth(), this.getHeight(), null);
g.dispose();
bs.show();
}
private void sleep() {
try {
Thread.sleep(33);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
public static void main(String[] args) {
Main game = new Main();
JFrame frame = new JFrame();
frame.add(game);
frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
frame.pack();
frame.setVisible(true);
game.start();
}
}
I think the confusion here comes from that cast to a DataBufferInt, the doc doesn’t provide any guarantee (that I can see) that this is a valid cast - and I imagine that if it is, they don’t intend for you to design your applications like since access to an internal data buffer that describes your image is something I can image the implementers of BufferedImage tried to avoid by returning the DataBuffer rather than the more powerful sub class. Is it stated anywhere that this is a guaranteed implementation?
That’s kind of a hack. It’s like casting a returned InputStream (whose underlying type is an IOStream) to an OutputStream.
Well, considering that DataBuffer is an abstract class I fail to see how that could be the case.
That argument makes no sense. Just because it returns an abstract class doesn’t mean you have the right to (safely) go around casting it to anything you want. If it does use an internal DataBufferInt - it breaks code design and stability by casting it and assuming that it will always be implemented as a DataBufferInt.
I think it is a pretty fair analogy with the IOStream
you have a class IOStream which imple,ents InputStream and OutputStream.
you have a function called getInputStream(), it returns an IOStream. They don’t have a getOutputStream because you aren’t guaranteed (or aren’t supposed to) write to the stream.
getInputStream() promises to return a reference to an inputstream. It returns a reference to IOStream. You see that and cast it to an OutputStream and begin writing to it.
3 years later, the code is refactored and it returns some exclusively readable resource that implements inputstream. You can’t possibly blame the implementing class - it never promised you that it would return an outputstream.
RGB, ARGB and ARGB_PRE BufferedImages will always be backed by DataBufferInt, but if you’re that bothered about it you can check it at runtime, or you can always build the BufferedImage from scratch to use an int[] and DataBufferInt.
Even if that is the case it is against the fundamental design principal of OOP code - whose to say you will always have write access to that buffer? They aren’t making any promises on those grounds. My implementation might use a more remote optimized cache that you can’t readily write to - what then? It is explicitly not providing you direct access to write permissions and you are hacking your way to them.
I really wouldn’t implement this - there are a lot of reasons for the underlying buffer to change.
That code can very easily break in later revisions of the JDK or when working on a different platform with a different jre. Don’t use it.
You’re absolutely right. They’ll go around radically changing old code breaking all previous applications that relied on it as opposed to building things on top of the old stuff or making completely unrelated code for new things. Just like, for example, Swing, NIO and JavaFX didn’t.
In any case, if you wan’t to make a game for a wide audience I suggest you use something like libGDX. Software pixel manipulation like this isn’t all that useful or effective anyway comparatively.
A heated discussion over nothing…
if(dataBuffer instanceof DataBufferInt) {
// pixel manipulation on int[]
}
else if(dataBuffer instanceof DataBufferByte) {
// pixel manipulation on byte[]
}
else {
// read pixels into own int[]
// pixel manipulation on int[]
// write int[] back into image
}
backed by DataBufferInt[list]
[li]BufferedImage.TYPE_INT_ARGB
backed by DataBufferByte
[li]BufferedImage.TYPE_BYTE_GRAY
backed by DataBufferShort
[li]BufferedImage.TYPE_USHORT_555_RGB
[/li]
[/list]
Yeah but what if they remove the DataBufferInt class altogether?! Did you think about that? Your code will break.
Non sense. Itis in the public API, they can’t remove it.
What if they remove java.lang.Object, what will we subclass? Did you think abut that? Your code will break.
Since when has anything been removed from the JDK?! ;D
Anyway, if you build the image in reverse from an int[] you can guarantee what’s backing it - such as here.
I love it when sarcasm goes unnoticed.