TDWorld Development Thread

User avatars now render above creeps, and creep health is represented by their size. As lasers shoot the creeps, they diminish into nothing. Eventually the creeps will be represented as balls of energy.

Buildings with towers are now transparent instead of just having a wireframe.

v1 of laser system, although the laser quads still don’t rotate to face the camera :sweat_smile:

Here the same avatar is used for everything, but once the new avatar generation UI is done I will have a better demo.

3 Likes

Lots of progress with the avatar and decal rendering pipeline today!

3 Likes

Recently, the game has been unplayable on my $2k phone (2fps with creeps, 15 without) during stress testing. It’s kind of an extreme scenario, but it’s also a scenario I expect, for example:

Also, when entering an area with lots of towers, sometimes FPS would drop from 50 to 5-10 for a couple seconds…

Updates (so far) today:

  • Removing copy-on-write data structures where I could.
  • When adding towers to scene, don’t re-render duplicate segments. (bug)
  • Lowering number of segments per tile from 100 to 64. This reduces the number of frustrum checks per tile, since we are not (currently) using a quad tree. This requires some fine tuning, as too large a segment and updating the game world will have noticeable lag, but too small and we waste CPU cycles checking if segments are visible. Likely I will be creating some kind of basic quad tree to solve this. We already have WorldTile->Segment, but likely we need a third layer.
  • Experimenting with not re-rendering duplicate route segments. Until we can split routes across tiles or tile segments, this is actually slower, so disabled until that other work is done.
  • Only make decals face camera when they, or the camera, changes rotation/position.

Result:

  • Now at 7fps in stress test with creeps, 50 without.

Currently, roads are only divided up by world tile (a slippy tile at zoom level 15), while buildings are divided up into segments that are 1/8th the size of the same slippy tile.

We could potentially gain some FPS by splitting the road polygons into these segments, which we only render when they are in the frustrum.

Creep routes and the creeps themselves are just added directly to the world, they are not “sharded” up into different segments, which will be the next step. We’re rending way more than we need to! :slight_smile: I expect this to raise the stress-test FPS from ~7 to 30. The goal is 60 on my fancy phone.

2 Likes

Update - splitting the creeps rendering up by segments (just hacked into building segment atm) gets us over 30FPS in our stress test! Yay!

More progress to be made. But a nice step.

5 Likes

So, to render the creeps, what I’m doing is:

modelBatch.render(modelInstance)

Is there a more efficient way for rendering something that is transformed every frame, in LibGDX?

EDIT: I noticed ModelBatch uses a RenderablePool without specifying the size, so it defaults to16. That means every frame I’m doing a few big allocations to grow the pool to a hundred/thousand in some cases.

Why isn’t it configurable? :frowning: I’ll have to create my own ModelBatch class with a one line change…

EDIT2: I suppose another change is to batch the creeps up into a single mesh like I do with buildings, but this might get unwieldy, I’ll have to figure out an abstraction to handle this cleanly if there isn’t a better solution.

Yesterday I got to work on this some. Using a ModelCache for the creeps, and some other tuning, I can get to around 20-25FPS in the stress test on my phone.

While this seems like a drop, it also includes properly moving creeps between segments, logic which currently executes in the rendering loop until I can move it out, and find the right abstractions to deal with concurrent access to certain resources.

In order to represent the creeps as balls of energy, I think I’ll just go with decals and see how well decals work with animations, by changing the texture every frame.

EDIT: An alternative of course is a shader, but will try the easiest thing first.

This should get us closer to the final design, and should also perform much better.

Once we can get sustained 50-60 FPS during my stress test, I’ll know I have a solid foundation and can move on to other things.

Update:

  • Moving creeps between segments is now off the UI thread
  • Lasers now don’t all fire at once, there was a bug where the last attack time was not initially sent from the server. This reduces the occasional framerate drop. I’m still trying to improve the laser rendering. They’re just quads, but they have a lot of impact on the framerate for some reason. Using a model cache does not help.
  • Various concurrency bugfixes and performance improvements

Still at around 20-25 FPS on my phone, but it’s more consistent now.

It caps out at 144 on my desktop.

Will work on it again Sunday. Today’s my birthday. :slight_smile:

3 Likes

Update:

30-40 sustained FPS in our stress test!

Still not 60, but closer!

I realized there was no reason to calculate what is visible each frame. So now, I do that on a separate thread. Additionally, my render loop consisted of a tree structure, since the world is made up of tiles->quads->segments->renderables. We had to iterate through this tree of classes during each frame, probably causing lots of cache misses.

Instead now, that separate thread iterates the tree structure and populates an array of items to render. The render loop just polls from this list, meaning that rendering simply is just iterating through one array.

This is managed by having two buffers of renderables - one used for rendering, while the other is prepared for the next frame.

EDIT: 60FPS now!

I disabled blending on the avatars, which I don’t need, and also made the avatar resolution lower depending on the screen size.

2 Likes

The creep bodies can now be animations, here’s a test with some random GIF I found:

I have to figure out how to make nice plasma-like animations in Blender, then we’ll have our own :slight_smile: And we can finally get around to making the tower attacks look nice too.

1 Like

Today lots of progress on the synchronization between client/server:

  • Instead of running a “similar” simulation on the client as the server (in regards to attacks), the server now sends the attacks that the client doesn’t yet know about, and the client just renders those. This improves the accuracy a lot and removes a lot of headaches for me that kept causing delays. If needed, we can always optimize it more - trying to duplicate the simulation is not worth the headache. This also saves a lot of CPU time / battery life on the client.
  • Creep positions are still “simulated” separately however, but this is much easier than dealing with that and attacks.
  • Fixed bug where eventually all the creeps on the screen would go away. This regression was introduced by our optimization to only render creeps on screen. Inside the CreepRenderable.render() method - we set a flag - “renderedLastPosition” to true. If this was false, we wouldn’t calculate the next position. This obviously no longer works with the new system - and without it the bug is fixed.
  • A new debug system was built so that on the client I can see which creeps are being rendered that are no longer on the server, and also I can tell the client to inform the server about how many creeps it knows about. This way, on the server I can randomly select clients to see how accurate the synchronization is, if need be, to detect bugs.

Upcoming:

  • Previously, creeps came from an object pool on the client. I removed that optimization while fixing the above. I need to add that back.
  • Remove scrolling from the sign up flow - right now the sign up UI is terrible - can’t launch until that is fixed.
  • Need to find someone to make animations for the creeps/lasers.
  • Ability for creeps to attack towers! :slight_smile:
1 Like

Much progress was made on attacks, server side and client side. This adds the ability to attack another tower using creeps. There’s still much more to do, as rendering the attacks is not yet complete, however everything looks to be working server side, the client gets the events, the creeps attacking the tower on the server will eventually destroy it, award the attacker, and stop the attack/spawning attack creeps.

On the client an issue with segments/quads flickering when they pop in/out of view was fixed by adding a delay:

public boolean isInViewDelayed(Frustum frustum, long now) {
    boolean visible = frustum.boundsInFrustum(topLeft.x, 0, topLeft.y, halfWidth, 0f, halfWidth);
    if (visible) {
        lastTimeNotVisible = 0;
        return true;
    }

    if (lastTimeNotVisible > 0) {
        if (now - lastTimeNotVisible > OUT_OF_VIEW_DELAY) {
            lastTimeNotVisible = now;
            return false;
        }
    }

    lastTimeNotVisible = now;

    return true;
}

I use the same technique for particle systems (which can be relatively expensive).

1 Like

Nice. We use this for the whole game world (it’s a tree).

Some progress today regarding attacking towers with creeps:

  • When the target is destroyed, the server properly stops spawning new creeps for that attack.
  • When creating a tower, you can choose the tower type in the UI now.
  • After initiating an attack, the map automatically deselects the attacking tower.
  • Fixed broken optimization preventing some information getting sent from server to client
  • When a tower is destroyed, ensure it actually gets removed from the clients +
  • When an attack completes, cleanup the attacking creeps.
  • For the last two items, the first pass of a new concept of “event” was implemented so the server can tell clients that certain things were deleted without having to send attacks that do the equivalent damage, for example.

Also, you can now see who you are logged in as in the game menu… :slight_smile:

Progress today:

The registration flow is now paginated, instead of just being a giant form, which is terrible to fill out.

A simple paginated UI framework was created on top of scene2d:

RequestCreateUser requestCreateUser = new RequestCreateUser();

root = new PaginatedStage(isHighRes, stage, new PaginatedView[]{
        new RegistrationUserNameView(isHighRes, requestCreateUser),
        new RegistrationFactionView(isHighRes, requestCreateUser),
        new RegistrationAvatarView(isHighRes, colorPicker, requestCreateUser),
        new RegistrationEmailView(isHighRes, requestCreateUser),
        new RegistrationPasswordView(isHighRes, requestCreateUser),
        new RegistrationSubmitView(isHighRes, api, requestCreateUser),
        new RegistrationWelcomeView(isHighRes, requestCreateUser),
}, hud::closeRegistrationUI);

This way I can easily maintain and reorder parts of the registration flow, as desired. It also supports things like interstitial views, which allows certain steps of the registration flow to own all logic in that step, like finalizing creating the user (submit view) without the user having to hit submit a second time.

public interface PaginatedView extends View {
    void setPaginationView(HorizontalGroup horizontalGroup);
    void validateAndShowErrors(PaginatedViewValidationObserver observer);
    default boolean isInterstitialView() { return false; };
    default void render() {};
}

Now that it’s functional, I’m working on making it look nicer and then I’ll put up a quick video of it.

2 Likes

Today was as busy day, so all that was done was:

  • Now able to sign up as Rogue, rather than being a specific faction.
  • The server properly handles conflicts between the different factions and Rogue users.
  • Validation server side for signup for the new faction, and some refactoring.
  • UI changes, and rendering the user’s faction in the game menu.

Next I’m planning on building a simple notification system, so that when the user creates links etc, they get little messages saying that the attack/trade route they setup is working.

Also, I am still looking for an artist/3d designer.

2 Likes

Notification system created!

tdworld-notifs

It created some fun challenges to solve.

Server-side, we don’t want to create contention/load on the producers that want to create notifications. This means, for example, we don’t want to handle the fanout when we actually create the notification itself.

Instead, we fire off the notifications right away into a queue. Currently, since the server runs on one big machine it gets put in a ConcurrentLinkedQueue.

A separate thread periodically goes through and 1. Pops items out of the queue and assigns them to a user and 2. Cleans up old notifications assigned to a user that were never consumed.

The client-side event loop on the server (which runs on multiple threads) periodically polls notifications for that particular user and pushes them to the client.

This gives us clear separation of responsibilities, and if we need to ever scale horizontally, this architecture lets us.

The impl. is fairly simple:

// BEGIN MANY THREADS
public void addNotification(ID userId, Notification notification) {
    notificationsInQueue.add(new NotificationQueued(userId, notification));
}

// If the server shuts off during the call of this method, some notifications will be lost.
// It would be nice if the client could say what notifications it got, and *then* we remove them from the server.
public void getUserNotifications(ID userId, List<Notification> result) {
    Queue<Notification> notifications = notificationsByUserId.get(userId);
    while (notifications != null && !notifications.isEmpty()) {
        Notification next = notifications.poll();
        if (next != null) {
            result.add(next);
        }
    }
}
// END MANY THREADS

// BEGIN PROCESSOR THREAD
private void bucketQueueByUser() {
    while (!notificationsInQueue.isEmpty()) {
        NotificationQueued notificationQueued = notificationsInQueue.poll();
        if (notificationQueued != null) {
            Queue<Notification> notifications = notificationsByUserId.get(notificationQueued.userId);
            if (notifications == null) {
                notifications = new ConcurrentLinkedQueue<>();
                notifications.add(notificationQueued.notification);
                notificationsByUserId.put(notificationQueued.userId, notifications);
            } else {
                notifications.add(notificationQueued.notification);
            }
        }
    }
}

private void cleanupOldNotifications() {
    long now = System.currentTimeMillis();
    for (Map.Entry<ID, Queue<Notification>> entry : notificationsByUserId.entrySet()) {
        entry.getValue().removeIf((notification) -> notification.createdAt + notification.lifeDuration >= now);
    }
}

public void processQueue() {
    cleanupOldNotifications();
    bucketQueueByUser();
}
// END PROCESSOR THREAD

We also have monitoring for if things like the processor thread taking too long.

The notifications are also persisted to disk for reliability etc. If you do something, like create a very long link, and then close the app, and then we do a deployment (restart server), and come back a week later, you’ll get the notification.

On the client, the notification UI framework is created in libgdx’s regular ol’ scene2d.

We now have notifications for when towers created, links are created, creeps start spawning for new links (attacks/trade routes).

Here’s a quick demo of the result!

https://www.youtube.com/watch?v=CyCDWZgmUpc

I have some plans to expand notifications, maybe after launch, where each notification can be tied to a geographical position, so you can click “see details” on the notification and jump there.

A refactoring of how the towers are linked has been done, which allowed me to resolve a bug where if a tower is attacked and destroyed, the routes to/from that tower on the client did not go away.

This also makes working with data in the backend much easier, as instead of storing to/from ids on each tower, it’s a directed graph, represented as a ConcurrentHashMap of TowerLink, sharded by our existing Shard mechanism. This makes reasoning about the data much easier and more efficient.

It also means that now I can add the ability to upgrade towers to the point that you can link a tower to/from many others, for example.

The link creation API has changed, but still simple, with the link to shard propagation encapsulated in TowerLink itself:

TowerLink towerLink = new TowerLink(towerA, towerB, null);
towerLink.propagateToShards(remoteGameState.getWorld());

RouterProvider.economyRouteRouter.enqueueForTower(towerA, towerLink);
remoteGameState.addNotification(
        new Notification(NotificationType.TOWER_ECONOMY_LINK_ENQUEUED)
);
1 Like

When selecting a building or tower, it’s nice to know, in the sea of buildings, what you’ve selected. Up to now, I was using a particle based animation that spun around the selected building.

This has been replaced with an arrow.

Also, towers now support health bars, and some race conditions on the client have been fixed to ensure the attacks finish playing before the tower is actually removed (the towers get removed a little sooner on the server than on the client).

https://www.youtube.com/watch?v=Vk1qj7-MQQA

Note: In the video things like the health bars & avatars don’t rotate to face the camera and movement is janky. This is just because of “God mode” on desktop needs some work.

This was done on Sunday, I was just lazy in writing the update.

Coming up next week:

  • Creeps not going away at end of attack?
  • Store last location on disk + show “outdated location” message
  • Camera to go to building on selection
  • Encryption for login/session info in transit

Some progress today - the Neutral building selection UI:

Lots more changes to come, like making history look nicer.

I had some ideas on making the stats look fancy, but going with this for now to get things done.

Includes a major refactor to how the building selection UI system works, so I can start churning out these UIs quicker.

1 Like