TDWorld Development Thread

First pass of geospatial query engine works! I’ve verified its accuracy down to a few feet.

Here is an example of the “optimized” search API where when you store an object with a range, it pre-calculates the nodes in the grid that that node can reach by calculating its bounding box, rather than doing the search every time. Also, it is not immutable like the RTree library. One downside of the RTree I was using was that every add or delete created a new tree, which meant constructing a new tree every time a creep moved, spawned, or reached its destination. This was still fairly fast, but a mutable data structure has the opportunity to be much faster (with the downside of having to deal with concurrency, but the engine handles that).

We also have the advantage of having to move creeps between “nodes” in this structure much less often, so less moving memory around and potential cache misses.

    GridIndex<PositionDouble> positionGridDB = new GridIndex<>(15); // 15 = slippy tile zoom level
    Tower tower = new Tower();
    tower.range = 30;
    tower.position = new Position(10, 10);

    PositionDouble positionA = new PositionDouble(10, 10);
    PositionDouble positionB = new PositionDouble(10.0 + PositionUtil.metersToWorldDegrees(29), (10.0 + PositionUtil.metersToWorldDegrees(29)));
    PositionDouble positionC = new PositionDouble(10.0 + PositionUtil.metersToWorldDegrees(35), (10.0 + PositionUtil.metersToWorldDegrees(35)));

    CellItem<PositionDouble> resultA = positionGridDB.addRanged(positionA, positionA, tower.range);
    positionGridDB.addRanged(positionB, positionB, tower.range);
    positionGridDB.addRanged(positionC, positionC, tower.range);

    List<PositionDouble> positions = positionGridDB.searchRangedMetersInCells(positionA, tower.range, resultA.cellIdsIntersecting);
    assertEquals(2, positions.size());
    assertTrue(positions.contains(positionA));
    assertTrue(positions.contains(positionB));

Here I’m doing a weird thing and storing the Position objects themselves in the structure. How it will work in the real world is a bit different - you have separate grids for Towers and Creeps, but they are joined. When you add a ranged item to the Tower grid, you get back the cells that the tower can reach. When you query the Creep grid, you pass in those cell ids to optimize the query, at the expense of memory.

Remaining work:

  • Switch back end to this new structure, with a feature flag.
  • Using feature flag, benchmark old vs new implementations.
  • Re-test client vs server accuracy
1 Like

Sounds cool. Will be great to see what the performance improvement is. I’ve always found that the graphics side is always the performance bottle neck and I work hardest on improving that. Sounds like you’re an expert with that already though and have squeezed a lot of performance out already.

Interesting that the old method based on the business geospatial code was so inefficient, throwing away the whole structure and re-building every time something changed.

One problem I stuffed up when trying to squeeze out performance from similar spatial code was to over-write each moving unit’s world coordinates every update frame.
The better way to do it is to update a transformation (translation and rotation) matrix every frame, and after subtracting the unit’s original world coordinates from the camera’s world coords, then apply the unit’s transform matrix to draw it.
That way you don’t get cumulating floating point errors which can be quite severe when the unit’s world (x,y) coordinates are say (10234123234.213, 32143221314.234) and the translation every frame is say 0.0000000001 which results in rounding problems when using floats or doubles, showing up as units that just don’t move because the least significant digits (the translation) are being ignored.
Cheers,
Keith

Hey Keith,

The old structure was fundamentally wrong for what I’m doing, and didn’t work. I could have forked it and fixed it, but I’ve been fighting it for a long time and the complexity in the library makes it hard to understand why simple bounding box queries return weird results.

The new structure is done, performs better from the extreme workloads I’ve tested (30k + creeps and thousands of towers in one square KM), and is actually accurate. I’m working on some charts that I’ll put up after work.

The graphics side has some bottlenecks too once the number of creeps goes above about 20k, but that is a separate issue. I already do the movement calculations on a separate thread, so for some reason rendering is slow.

1 Like

Here’s the benchmark of tick times between the R-Tree and my custom Geohash index.

As you can see, creep traveling is faster but searching for creeps from a position is a little slower in some cases, with an overall improvement. This makes sense, since with the RTree library there is more overhead in creating a new tree with moving a creep vs with just updating some data in the Creep object in my library, until the creep moves out of its current “hash”.

I’m also using a ConcurrentHashMap and a CopyOnWriteArrayList to be extra safe, but I may do some testing and see where I am accessing the shard across threads - those cases are the minority and probably I can just add some locks in the “cold” paths and get rid of the CopyOnWriteArrayList.

Below 10k creeps in a tile, my implementation is faster. Beyond that right now, the performance falls off. That’s not really a problem as that’s an extreme case. I still have some optimization I can do, but I think I’m ready to move on to getting the beta out.

Remember that our original goal was accuracy, and performance was a stretch goal. So I’m glad we have almost equal performance, with the opportunity to go further.

2 Likes

Here’s how I do my performance testing. I adjust the NPC seeding to seed every building in the area. :slight_smile:

2 Likes

Working on my “nautical” to “game-world” space translation bounding box checks, created a fun to look at visualization. You can see toward the bottom of the map the bounding box checks return the wrong result. Debugging that. This is all part of the “get the attacks on the server and client synchronized”.

If we increase the number of testing points, we can see more patterns (as well as the problem not just near the bottom of the map):

EDIT:

We can look at the data and see that latitude normalization is broken - oops!

lat: 90.0 lng:180.0 32767.0 -5188.264000000003
lat: 80.0 lng:180.0 32767.0 3678.4840000000004
lat: 70.0 lng:180.0 32767.0 7333.518292682926
lat: 60.0 lng:180.0 32767.0 9515.18472222222
lat: 50.0 lng:180.0 32767.0 11113.903887688986
lat: 40.0 lng:180.0 32767.0 12405.718041704444
lat: 30.0 lng:180.0 32767.0 13519.735417919423
lat: 20.0 lng:180.0 32767.0 14525.581378163679
lat: 10.0 lng:180.0 32767.0 15469.878889378582
lat: 0.0 lng:180.0 32767.0 16385.0
lat: -10.0 lng:180.0 32767.0 17298.121110621418
lat: -20.0 lng:180.0 32767.0 18242.41862183632
lat: -30.0 lng:180.0 32767.0 19248.264582080577
lat: -40.0 lng:180.0 32767.0 20362.281958295556
lat: -50.0 lng:180.0 32767.0 21654.096112311014
lat: -60.0 lng:180.0 32767.0 23252.81527777778
lat: -70.0 lng:180.0 32767.0 25434.481707317074
lat: -80.0 lng:180.0 32767.0 29089.516
lat: -90.0 lng:180.0 32767.0 27577.735999999997

2 Likes

Yay!


(blue dots mean match is correct)

Trying out a different algorithm. Looks good so far. Have to finish fixing my tests from ripping out the old RTree library and then will see if client can predict attacks on server within 10% yet.

5 Likes

Nice.
java.lang.ArrayIndexOutOfBoundsException: 420

Update video coming soon.
Another milestone reached.
Attacks and creep movement between server and client is within 1% accuracy (tested over 30mins so far). Only took working from like 9AM to 8:30PM on a Sunday…

[com.winricklabs.tdworld.util.CreditSyncDebugger] Credit delta, Server=[1500] Client=[1495] Accuracy=[100.33444816053512]
[com.winricklabs.tdworld.util.CreditSyncDebugger] Credit delta, Server=[1500] Client=[1500] Accuracy=[100.0]

Creep movement has also been “smoothed” on the client. Before it would appear that the creeps moved one “step” and then stopped and moved again. The interpolation has been improved, so there’s no stopping now.

2 Likes

Some days are like that!

1 Like

Indeed!
Up next is testing building for Android and iOS, and adding the “location based” aspect so that moving your phone around changes your view into the world.

1 Like

Trying to build for Android, anyone ever get this?
I’ve tried clearing Gradle cache, Invalidate/Restart IDE, played with the activity class path…

Activity class {com.winricklabs.tdworld/com.winricklabs.tdworld.AndroidLauncher} does not exist.

Verified these items: https://github.com/facebook/react-native/issues/14952#issuecomment-314683712

Tried my other computer. I can’t get gradle to run the Android build with Intellij 2020.2, Gradle 6.1.1., and Java 1.8.

I get this:

Could not open cp_init remapped class cache for bl3ndxec306ta9f2jnqm0ilm3 (C:\Users\winri\.gradle\caches\6.1.1\scripts-remapped\ijresolvers_82otrqblfrs7r0ll85yaxma37\bl3ndxec306ta9f2jnqm0ilm3\cp_init3607aee355f62839c5e6f549478ccc87).
Could not open cp_init generic class cache for initialization script 'C:\Users\winri\AppData\Local\Temp\ijresolvers.gradle' (C:\Users\winri\.gradle\caches\6.1.1\scripts\bl3ndxec306ta9f2jnqm0ilm3\cp_init\cp_init3607aee355f62839c5e6f549478ccc87).
Could not initialize class org.codehaus.groovy.vmplugin.v7.Java7

Gradle/Java version:

C:\Users\winri\Documents\GitHub\tdworld-libgdx>gradlew --version

------------------------------------------------------------
Gradle 6.1.1
------------------------------------------------------------

Build time:   2020-01-24 22:30:24 UTC
Revision:     a8c3750babb99d1894378073499d6716a1a1fa5d

Kotlin:       1.3.61
Groovy:       2.5.8
Ant:          Apache Ant(TM) version 1.10.7 compiled on September 1 2019
JVM:          1.8.0_265 (Eclipse OpenJ9 openj9-0.21.0)
OS:           Windows 10 10.0 amd64

One piece of advice was to rebuild the android submodule. Trying to do so via right click -> rebuild in Intellij gives me:

Execution failed for task ':core:clean'.
> java.io.IOException: Unable to delete directory 'C:\Users\winrid\Documents\tdworld-libgdx\core\build'
    Failed to delete some children. This might happen because a process has files open or has its working directory set in the target directory.
    - C:\Users\winrid\Documents\tdworld-libgdx\core\build\libs\core-1.0.jar
    - C:\Users\winrid\Documents\tdworld-libgdx\core\build\libs

Not sure yet what is using files in that directory.
It feels like this stuff is fighting me every step of the way…

EDIT: Was able to stop the gradle daemon:

 WMIC PROCESS where "Name like 'java%' AND CommandLine like '%GradleDaemon%'" Call Terminate

Then delete the “build” directory with Intellij closed.
Now it builds with the warning:

WARNING: Compatible side by side NDK version was not found. Default is 21.0.6113669.
Compatible side by side NDK version was not found. Default is 21.0.6113669.
Unable to strip the following libraries, packaging them as they are: libgdx-box2d.so, libgdx-bullet.so, 
libgdx-freetype.so, libgdx.so.
Compatible side by side NDK version was not found. Default is 21.0.6113669.

EDIT
And it still doesn’t run, I still get:

Starting: Intent { cmp=com.winricklabs.tdworld/.AndroidLauncher }
Error type 3
Error: Activity class {com.winricklabs.tdworld/com.winricklabs.tdworld.AndroidLauncher} does not exist.

Thanks to myke on the libgdx Discord server I found that it does work via the terminal:

gradlew android:installDebug android:run

Why must you do this to us Intellij/Gradle?

Anyway after that I got this fun error at runtime:
java.lang.NoSuchMethodError: No static method metafactory(Ljava/lang/invoke/MethodHandles$Lookup;Ljava/lang/String

Had to go back to Java 1.8 in my system’s PATH… when will libgdx support a version newer than 1.8, I wonder.

As an Android developer myself, I can say that it has nothing to do with LibGDX. It is caused by the Android SDK itself which doesn’t support Java 9+. They might have updated, but then there is a court case running between Oracle and Google and hence they couldn’t update.

If you must use Java 9+, I suggest trying out any AOT solutions (Azul has got one I guess, not sure) and run it using NDK (Native Development Kit). Another easier option is to move to Kotlin, so you can have all the latest language features and yet compile to native SDK.

Great follow up and information, thanks!

1 Like

The game is now fully location based, with the camera at the ground, and orientation based on the device, for Android.

Going to get iOS working next, then likely will be working on the animation to zoom out to strategic view, and finally starting to put the UI together.

2 Likes

I’ve noticed a pretty frequent flicker on Android, which I’m thinking is the GC, so investigating that today hopefully.

The camera can now be animated into different positions! This will be used for jumping out to “strategic” view:

            Vector3 position = new Vector3(currentPosition.x, cameraY, currentPosition.y);
            Vector3 endPosition = new Vector3(currentPosition.x, 3.5f, currentPosition.y);
            float animationTargetRunTime = 2000f;
            animationLoopRunner.addJob(new AnimationJob(animationTargetRunTime, (duration, isDone) -> {
                position.interpolate(endPosition, duration / animationTargetRunTime, Interpolation.smooth);
                cameraY = position.y;
                perspectiveCamera.position.set(position.x, position.y, position.z);
                perspectiveCamera.lookAt(currentPosition.x, 0, currentPosition.y);
                if (isDone) {
                    handleViewPortChange();
                }
            }));
1 Like