TDWorld Development Thread

[HappyHydra]

1. Today the client/server work for making renaming towers functional was done. If you purchase and apply a Custom Tower Name module, you can rename that tower. Also, removing that module reverts the tower name back to whatever the map provider says it is (or, “Un-named”).

Profanity checks will be added before launch…

2. The Store/Inventory UI was improved, so now when switching tabs the window stays the same size. Additionally, the Inventory shows the empty product slots, and clicking one will go back to the store.

3. Another change you can see here is the scrollbar, which was added to the common Modal component.

4. The modal for adding/removing modules from towers is now a generic “Product Details” modal, and displays information about the product:

5. The Login/Registration flows have been converted to the new Modal component, which doesn’t change appearance much, but makes working with the code easier.

6. A bug in the client reconnect logic was fixed which resolves not being able to always reconnect after network/server issue. Also, a loading animation was added for when reconnecting.

7. The modal backgrounds were darkened a bit to improve readability. These are still images, but in the future I’ll probably convert to images generated at runtime since now I have those tools. This will allow faster iteration to improve the appearance (and slightly decrease download size! ).

[HappyHydra]

Infrastructure around the backend being able to send emails is done, and the first pass on the email verification flow is functional.

[HappyHydra]

Just as an update, yesterday the creep bodies were changed to not use someone else’s animation we’re now using a generated animation from a tool, but I’m not super happy with it yet.

[HappyHydra]

Note that this project is not abandoned. I am simply switching to another project to finish some goals before EOY.

[HappyHydra]

I’m still thinking about this project! Hope to return in coming months. Been making great progress on another project that I want to “finish”.

[HappyHydra]

I’m baaaack!

Update today:

• In preparation for launch, security is being improved a bit. Passwords are now hashed on client using RSA-256, so user passwords aren’t sent over the network or stored on the server.
• Exploring using SSL for only low-volume high-risk messages.
• The user’s last GPS position is now stored locally, so the game world can quickly resume on re-open, or if GPS isn’t yet acquired on relaunch. Much better than a black loading screen.
• A “Waiting for GPS” animation has been added.
• When an account is created, you now start with 100 credits. As long as you are logged in, you will always passively earn 10 credits a minute.

Next:

• Forgot Password flow.
• A “Welcome Back” modal, which shows stats, etc. Also, users are rewarded for returning to the game after a period of time.
• Ability to purchase credits.
• Improved faction icons.

[HappyHydra]

Update:

• While not super sexy work, Password Reset is done and tested.

[HappyHydra]

Update:

• Server Optimization: Request handler for server no longer uses a Map of enum to RequestHandler, and instead uses a generated switch statement (which will be much faster).
• Server Optimization: GridIndex (used for geospacial lookups) used a CellID which was backed by a String representation of coordinates. This was changed to just a long which has the coordinates packed inside. This way, when we do a lookup in the internal Map in the GridIndex we just search by a long value, and not having to hash a string of coordinates… This also makes serialization/saving faster.
• Server Optimization: Switched some uses of ArrayList<boxed primitive> to libgdx’s array classes.
• Server Optimization: Removed usage of some types which were added to add some type safety, like Point2D, GeoPoint2D, etc, from some “hot” areas of the codebase, like with collisions and building geometry merging. Tradeoff of type safety vs performance in these areas (removes extra allocations etc).

[HappyHydra]

Update:

• Optimization: Switched to FST for on disk and over-network serialization of data. Up to now was just using Java’s object serialization which is nice in some ways but definitely not optimal in terms of size or CPU usage
• Upgraded backend to Java/JDK 16->17 and finally upgraded from Gradle 4 to 7.3.
• Fixed bug with new GPS loading indicator showing above everything and just in general being annoying.
• Working with artists to create art for the different factions.

[HappyHydra]

Update:

• Looks like FST won’t work, as it does not allow old clients to read newer objects, even with @Version annotations, AFAIK.
• Looking into potentially using CaptainProto. I’ve used Thrift in the past but looks like CP might be better for my use case.

[HappyHydra]

Update:

• Going with very traditional serialization mechanism.
• Not using Java built in object input/output streams.
• This avoids runtime reflection type detection etc which is slow.
• Thrift has big memory allocation issues.
• Captain Proto claims to be copy free, but the fact that it’s too slow to use at runtime in the rest of the app means I have to construct the Proto objects to send each time anyway, so it still results in copies.
• GRPC is neat but too big of a change. Also, is unnecessary overhead anyway vs what we can achieve with a little work.
• Rolling my own serialization mechanism. This way, if an object only needs four bytes, we only send four + four (int message type) bytes and not the 300+ that Java wants to send…
• We handle backward compatibility by making our APIs append-only, which is very “normal” way of doing things.
• This also allows us to reuse the buffer from the client, meaning zero buffer copy on message handling, and we can reuse the buffer for sending messages to each client.

Just had to create some of our own classes: DataFrame and OutputDataFrame.

Fun fact: OutputDataFrame actually uses libgdx’s ByteBuffer underneath (in the future we will be able to swap this out for a stream).

Example:

public class AvatarDefinition {
    public AvatarType avatarType;
    public int[] colors;
    public int[] parameters;

    public static AvatarDefinition deserialize(DataFrame dataFrame) {
        AvatarDefinition avatarDefinition = new AvatarDefinition();
        avatarDefinition.avatarType = AvatarType.values[dataFrame.readInt()];
        avatarDefinition.colors = dataFrame.readIntArray();
        avatarDefinition.parameters = dataFrame.readIntArray();
        return avatarDefinition;
    }

    private void serialize(OutputDataFrame dataFrame) {
        dataFrame.writeInt(avatarType.ordinal());
        dataFrame.writeIntArray(colors);
        dataFrame.writeIntArray(parameters);
    }

I expect this conversion to be done by next week, then it’s back to load testing and working on game loop.

EDIT Note: This is one reason I love Nim. If I was using Nim, I could just do this all at compile time I suppose I could also do it compile time here via some build scripts, but definitely not as nice.

[HappyHydra]

To get the game out faster, I’ve made the following decisions:

• All artwork and story related elements will be added after launch. Launch will only contain core gameplay.
• For launch we will remove the concept of Faction and all users will play as Rogue until later.
• For launch we will allow users to play without creating an account. We will create a “shadow” account, and the user can convert this to a “real” account with their own username/password later.

We still have to do quite a few things before launching, but this makes things easier.

[HappyHydra]

Update:

• Conversion to new serialization system is complete! Only like 5k lines and 180 files changed…
• Up next is removing the requirement to create an account to play, with the ability to “complete signup” later.

[HappyHydra]

Update:

• Serialization system progress: bugfixes etc.
• Got to benchmark new serialization system. Seeing about 3 8kb spikes when a user first connects, and then about 100 bytes every few seconds with a couple creep routes going. Pretty happy with that.

Still working on making it so users don’t have to create an account to start playing the game, and then we will be doing benchmarking to ensure the server can handle 100k clients for launch.

[HappyHydra]

Progress today:

• Users can now play without creating an account. The backend work was completed for “Shadow accounts” and migrating these to actual accounts. This works based on generating and storing a session key on the device.
• This involves generating avatars and usernames on the server, which is done.
• A functional version of the welcome UI for the shadow accounts is done, hope to complete next week.
• This all will help conversions and getting more users on launch.

Also:

• The serialization system was updated to support extending objects easily and be backward compatible (this was supposed to be in the original design but didn’t work due to bugs).

[HappyHydra]

Progress today:

• Shadow (temporary) accounts can now be easily upgraded to real accounts via the game menu.
• Shadow users can install the app and play right away with a generated avatar/username.
• The Welcome UI for the shadow accounts and game menu is functional.

Game menu right now:

[HappyHydra]

Second pass on creep animations done! Going with 3d spinning decahedron with user’s avatar inside for now.

TDWorld CreepsV2 Animation Test

Also, when you switch to strategic view the creep’s “body” now goes away and you just see the avatar.

Today was not a good day for recording. Using Windows screen recorder, Icecream Screen Recorder, OBS, and V Recorder for Android all gave me trouble so just recorded the damn screen with my phone…

Up next is:

• Tower laser animation update
• Build load test tool for server
• In-app purchases
• Get building w/ iOS

[HappyHydra]

Update today:

• Tower laser animations updated.
• Rewrote multithreaded render pipeline to fix flicker issues by using two queues to keep track of buffer ready to be rendered and buffers ready to be reused for determining what can be rendered.

Up next is to load test the server.