JBox2D-Networking, Reproducibility and Determinism

[klaus]

So we are developing an online multiplayer game using JBox2D for physics. I’m also using the lockstep mechanism to keep everything in sync. It actually works really well on our tested systems, but I worry a bit about optimized floating point operations on different systems.

My research on this topic suggests that all modern CPUs use the SSE2 instruction set for floating point ops and that would mean that all calculations are processed in the exact same way on those CPUs.

But the question remains, can I be sure somehow? A (possible) different instruction set in the future could break the reproducibility and therefore my network implementation.

Using strictfp is not an option, because of the significant performance hit you get from using it.

[basil]

no, you cannot be sure at all. even with strictfp : assuming you do not run a fixed timestep.

the usual approach to this issue is to synchronise the full state at a low frequency, compute “corrections” and interpolate current state towards the correct state.

[Abuse]

[quote]Using strictfp is not an option, because of the significant performance hit you get from using it.
[/quote]
Have you actually benchmarked this assumption?
My experience is that, so long as you’re not mixing strictfp & non-strictfp code, there’s zero performance impact.

[klaus]

@basil

no, you cannot be sure at all. even with strictfp : assuming you do not run a fixed timestep.

I do use a fixed timestep.

[quote]the usual approach to this issue is to synchronise the full state at a low frequency, compute “corrections” and interpolate current state towards the correct state.
[/quote]
I’m not 100% certain that I fully get you here, but I think you mean sending the correct state from the server to the client and then correct the client’s position. Are you talking about client-side prediction? If so, this is exactly what I do not want to do and this syncing is actually the reason I want to use lockstepping. We used client-side prediction before, but ran into a lot of problems when having to sync 100+ entities. (RTS style of game)

[quote=“Abuse,post:3,topic:54755”]
I did. See my test class here: http://pastebin.com/xRhd9kaf

Results running this test on my PC:
TIME: 1001
TIME STRICT: 1785
TIME: 923
TIME STRICT: 1630
TIME: 980
TIME STRICT: 1245

Results running this test with the small adaption you proposed when using strictfp for the whole class:
TIME: 1873
TIME STRICT: 1385
TIME: 1276
TIME STRICT: 1263
TIME: 1315
TIME STRICT: 1243

[Abuse]

[quote]I did. See my test class here: http://pastebin.com/xRhd9kaf
[/quote]
…and the Float class is strictfp too?

If not, all your benchmark confirms is that there is a significant overhead when transitioning between strictfp & non-strictfp.

[theagentd]

@Abuse

[quote]I did. See my test class here: http://pastebin.com/xRhd9kaf

…and the Float class is strictfp too?

If not, all your benchmark confirms is that there is a significant overhead when transitioning between strictfp & non-strictfp.
[/quote]
This test isn’t good at all. It’s a Float instance instantiation benchmark. I’m not that good at microbenchmarks, but I’m fairly sure this at least is a better test:

package engine.test;

public class StrictFPTest {
	public static void main(String[] args) {
		
		int iterations = 0;
		long normal = 0;
		long strict = 0;
		
		long start;
		float f = 1;
		while(true){
			
			start = System.nanoTime();
			f += test1(f);
			normal += System.nanoTime() - start;
			
			
			start = System.nanoTime();
			f += test2(f);
			strict += System.nanoTime() - start;
			
			iterations++;
			
			if(iterations % 1 == 0){
				System.out.println("normalfp: " + normal/iterations / 1000 / 1000f + "ms");
				System.out.println("strictfp: " + strict/iterations / 1000 / 1000f + "ms");
				System.out.println(f);
			}
		}
	}

	static float pi = (float)Math.PI;
	
	private static float test1(float f) {
		for(int i = 0; i < 32*1024*1024; i++){
			f = (f + pi) / (f * pi * pi);
		}
		return f;
	}

	private static strictfp float test2(float f) {
		for(int i = 0; i < 32*1024*1024; i++){
			f = (f + pi) / (f * pi + pi);
		}
		return f;
	}
}

Gives pretty much equal performance for both of them.

[klaus]

@Abuse

[quote]I did. See my test class here: http://pastebin.com/xRhd9kaf

…and the Float class is strictfp too?

If not, all your benchmark confirms is that there is a significant overhead when transitioning between strictfp & non-strictfp.
[/quote]
You sir are correct and I’m an idiot ;D Thanks!

Adapted my test: http://pastebin.com/sXFRM5by

Output:
TIME: 152
TIME STRICT: 155
TIME: 144
TIME STRICT: 142
TIME: 156
TIME STRICT: 143

:

Mkay… and I found something interesting too. Because I wanted to have fast Math operations I used the Apache Commons Math3 FasthMath class.

Slooooow:

	public strictfp static float distanceStrict(float x1, float y1,
								   float x2, float y2)
	{
		return (float) FastMath.sqrt(distanceSqStrict(x1, y1, x2, y2));
	}

Fast:

	public strictfp static float distanceStrict(float x1, float y1,
								   float x2, float y2)
	{
		return (float) StrictMath.sqrt(distanceSqStrict(x1, y1, x2, y2));
	}

Must be the conversion again

So another problem pops up: You have got to annotate every class with strictfp… I already searched for a more elegant solution to this but found none. So every library I use I have to grab the source code and annotate every class. Of course I would need to add that only to those that do floating point ops, but I’d be faster to just use an automatic search and replace on all classes probably.

Isn’t there a way to load every class with strictfp? Maybe a custom classloader? Or a compile option? JVM option? (only in a non-official JVM apparently)

[quote=“theagentd,post:6,topic:54755”]
You are of course correct too.

[Abuse]

Use a javaagent, along with a bytecode instrumentation api.

The javaagent has to be packaged in a jar, with a manifest that includes the property:

Premain-Class: some.package.Agent

…and the jar supplied to the jvm using the -javaagent command-line parameter.

Using ASM, it goes something like this:
(For succinctness I’ve used anonymous classes)

package some.package;
import java.lang.instrument.ClassFileTransformer;
import java.lang.instrument.IllegalClassFormatException;
import java.lang.instrument.Instrumentation;
import java.security.ProtectionDomain;

import org.objectweb.asm.ClassReader;
import org.objectweb.asm.ClassVisitor;
import org.objectweb.asm.ClassWriter;
import org.objectweb.asm.MethodVisitor;
import org.objectweb.asm.Opcodes;
public class Agent {
	
	public static void premain(String agentArgs, Instrumentation inst) {
		
		inst.addTransformer(new ClassFileTransformer() {
			
			@Override
			public byte[] transform(ClassLoader loader, String className, Class<?> classBeingRedefined, ProtectionDomain protectionDomain, byte[] classfileBuffer) throws IllegalClassFormatException {
				ClassReader cr = new ClassReader(classfileBuffer); 
				ClassWriter cw = new ClassWriter(cr, 0);
				cr.accept(new ClassVisitor(Opcodes.ASM5, cw) {
					@Override
					public MethodVisitor visitMethod(int access, String name, String desc, String signature, String[] exceptions) {
						if((access&Opcodes.ACC_ABSTRACT)==0) {
							// According to the JVM spec, it is illegal for abstract methods to have the strict flag set.
							access|=Opcodes.ACC_STRICT;
						}
						return super.visitMethod(access, name, desc, signature, exceptions);
					}					
				}, 0);

				return cw.toByteArray();
			}
		});
	}

}

[Abuse]

@KaiHH

Just one note: Setting the flag on the class itself would also work:

        cr.accept(new ClassVisitor(Opcodes.ASM5, cw) {
        	@Override
        		public void visit(int version, int access, String name,
        				String signature, String superName, String[] interfaces) {
        			super.visit(version, access | Opcodes.ACC_STRICT, name, signature, superName, interfaces);
        		}
        }, 0);

Not according to the class file format specification.

The class file format specifies ACC_STRICT valid as a Method access_flag only, not a Class access_flag.

The ability to flag a whole class as strictfp in source code is purely a syntactic convenience.

[KaiHH]

Darn… You are totally right. Sorry.
I should have clicked on that ACC_STRICT field in the IDE to see this in ASM’s source code:

int ACC_STRICT = 0x0800; // method

Sorry, for posting. I’ll delete that.

[Abuse]

@KaiHH

Darn… You are totally right. Sorry.
I should have clicked on that ACC_STRICT field in the IDE to see this in ASM’s source code:

int ACC_STRICT = 0x0800; // method

Sorry, for posting. I’ll delete that.

Don’t feel bad, I made the exact same mistake myself when I first implemented my StrictifingVisitor ;). (some time ago for a different project)

[klaus]

[quote=“Abuse,post:8,topic:54755”]
That’s awesome! An agent didn’t came to my mind! Thanks a lot, that’s brilliant! 8)

Oh, I feel so relieved - that topic was on my mind for a while now!

[mucaho]

It would be helpful to mention that there is already an issue for this exact reason on JBox2D’s github page.
This information in here is invaluable, especially on-the-fly adding of strictfp, as having two different codebases for a non strictfp and a strictfp JBox2D variant or having only the strictfp variant was dismissed by the library’s author.

Determinism of floating point is the only real benefit for developing in Java in my opinion, I hope this gets added to JBox2D.