Component Systems: Artemis style systems vs. traditional fat entities

[Roquen]

I’m ignoring the you’re working on middleware for the moment.

@Catharsis

“data oriented”… what I meant was a metrics based approach. IE actually measuring the efficiency of various designs.

I’d say data oriented is a cache oblivious (or conscientious) approach to data layout based on how data is transformed (access patterns). It also pretty much independent of other design choices…likewise for data driven.

Splitting data from logic at any level of an architecture allows one to focus on how to access it and better organize it for processing.

Sure, nothing to do with component based.

It allows one to query a component manager (CM) / container and get back an iterator over all components stored by their meta-type (all data, all systems).

…and store it in a tightly packed array or perhaps byte buffer or whatever data structure that is the low level DOD form.

Performing a query for something you know. (again ignoring middleware).

@Roquen

The whole component based thing is really rooted in modern “best practices/design pattern-y” think…

Is that a bad thing? For JGO it seems like a poison pill for more than a few.

Yes it’s a insanely bad thing. I’m not sure what you’re referring to as a poison pill however.

@Roquen

aka every thing you write is a nice reusable generic mini-library. That’s the only situation where it makes sense.

Isn’t that the promise, traditional OOP in the 90’s included, that is sold to management regarding adopting new principles and technology.

The big difference is that the traditional OOP direction didn’t deliver on this promise. Implicit CA + EventBus does.

Yeah and the goal of that promise it to service the bulk of programming jobs which is for project with much longer lifetimes than engineer turnover rate. It the most expensive method in terms of both engineering and CPU time but the reduction in risk makes it worthwhile from a business standpoint. But this isn’t something to focus on in education or literature because it’s roughly the equivant to coding conventions. Anybody that can program can pick it up in approaching zero time.

This way you can pass structures implicitly between modules without leaking a ton of concrete types and poisoning the larger codebase through unchecked dependencies.

Ok and there are tons of ways to achieve this.

@Roquen

Flip it around into type-think. When in a specific game do you need a type which can contain an arbitrary number of arbitrary types? Answer: never.

! When an artist is in control and not a coder.

The game design is in control. Coders, artist and worldbuilder should be in control of their tasks. I don’t disagree with any of your bullet points.

@Roquen

For example I frequently have an entity type which can logically contain an arbitrary number of (by name) variables where are logically typeless but are concretely one of a very small number of types (like entity, int, float, string). Data segregation (for data oriented) might be handled by monomorphic access to hide implementation details at the scripting level.

Type safety is handy for many reasons.

In general purpose programming, yes, you want to insure correctness as quickly was possible. However the user here is the scripting side which personally I want to be as fault tolerant as possible. One could go with typed, but I don’t since typeless is slightly lower complexity IMO.

I’m curious about the last sentence though regarding data segregation and “monomorphic access” in regard to a scripting interface because the latter is not clearly defined. “Monomorphic access” brings up connotations of immutability or marking classes or accessor methods final. Could you explain this more?

Any instance method which doesn’t override anything and is final is monomorphic at all callsites (places in code where it’s invoked). The runtime compiler can then treat it as a static method and all optimizations possible for statics are opened up (notable inlining). If not inlined then it’s a direct call. HotSpot can optimistically determine if any instance method call is monomorphic. If proved wrong later it will deoptimize. For completeness it also detects bimorphic (exactly 2 versions) and in that case can covert the invocation to an ‘if-then-else’ construct. Three or more will go through the virtual table.

@Roquen

Don’t write middleware if that isn’t your product.

Phew, glad this is what I’m doing and is an approved machination.

Of course the rules are different for middleware. I don’t talk about that because I prefer the not encourage middleware/library creation.

[junkdog]

@Catharsis

I suppose that brings up an interesting distinction in classifying ES. None of this is rock solid terminology, so I’m curious if folks have any input. I’ll classify Artemis / Ashley as type I ES. Type I basically lending themselves to having distinct structure such as having specific classes like “EntityManager”, “Entity”, “EntitySystem” where processing of entities occurs sequentially in a fully composed state. A type I ES often has hard coded classification schemes baked into the API that can’t be altered. This is the aspect / family classification in Artemis / Ashley. In this case the CA elements that may be marginally present are hampered by a hard coded relationship to the classification scheme.

What exactly do you mean by processing of entities in a “fully composed state”? In both artemis and ashley, entities are only a little more than glorified integers (entityId - and some additional metadata, not necessarily stored by the entity itself). Or is the hint in your definition of “Type II Es”, being “fully generic” and having “self-modifying capabilities” (components as more than data-holders, I guess) - not entirely sure what either implies.

@seismic

Mhhh okay…Still not too convinced here I guess. Let’s wait for that performance benchmark ;D ;D
(Not that it matters too much because Artemis is already pretty fast? So if you want to use an ES you already have a pretty good option there imo)

https://github.com/junkdog/entity-system-benchmarks - Nothing complete, but it’s a start.

[Roquen]

I keep forgetting. The argument “OO isn’t good enough so I use component based” is flawed. Component based is OO.

[delt0r]

Well i always think these debates are pretty Meh. But i still read them.

I mostly had issues with what problem Entity stuff solved, as in i couldn’t see the point all that much. I assumed because i didn’t understand them. Turn out i did. Also turns out when the right problem came along then it did become clear what they are good for. I should also note i didn’t use it for performance, since i got a few clusters i can use with over 5k cores. But it made the code small and fairly compact.

For smaller indie games however its less clear how it helps. Its up there with writing an engine. Write a game, not an engine. No one else is going to use it anyway.

[Roquen]

It would be much more interesting to have a discussion about various code design structures for these types of issues and the various trade offs. I’ve never run across a situation where moving past mimicing an open-class was needed. I’d actually avoid talking about data-oriented as it’s independent and starts to bog things. Actually data-oriented is way more important (assuming sufficient scale).

[princec]

Consider an RDMBS engine that does for physical data layout what Hotspot does for bytecode compilation.

Cas

[Catharsis]

@Roquen

I’d say data oriented is a cache oblivious (or conscientious) approach to data layout based on how data is transformed (access patterns). It also pretty much independent of other design choices…likewise for data driven. … Sure, nothing to do with component based.

We’ll go with “conscientious”… It can be independent, but if the larger architecture naturally supports data oriented concepts it’s easier to apply them broadly.

@Roquen

It allows one to query a component manager (CM) / container and get back an iterator over all components stored by their meta-type (all data, all systems).

…and store it in a tightly packed array or perhaps byte buffer or whatever data structure that is the low level DOD form.

Performing a query for something you know. (again ignoring middleware).

You’re thinking about it the traditional way…

While I’m still adding stream / filtering type mechanisms into the CA. The idea is to be able to do a query like “give me all the data components that also have the @SerializeMeOnNetwork annotation” in order to serialize an entity or whatever and send it over the wire. This implementation is a little less general purpose than the java 8 streaming API. I’ll probably limit it to type and annotation based queries. Maybe some bespoke processing though I’m keen to keep things fast on Android.

It makes possible generalized systems of a larger architecture while maintaining reasonable speed. Serialization is one of the larger use cases from a design perspective. On my side.

@Roquen

@Roquen

The whole component based thing is really rooted in modern “best practices/design pattern-y” think…

Is that a bad thing? For JGO it seems like a poison pill for more than a few.

Yes it’s a insanely bad thing. I’m not sure what you’re referring to as a poison pill however.

I get a ton of knowledge through following general thought leaders. Most recently in the past ~3 years the whole mechanical sympathy scene / re: Martin Thompson and co. Turns out a lot of what the HFT community is doing has applications to the game dev / interactive media scene.

I think you may be complaining that historically the “best practices / design pattern” scene for Java has been rooted in the enterprise side of things where there is mostly dubious value to game dev, etc.

@Roquen

Isn’t that the promise, traditional OOP in the 90’s included, that is sold to management regarding adopting new principles and technology.

The big difference is that the traditional OOP direction didn’t deliver on this promise. Implicit CA + EventBus does.

Yeah and the goal of that promise it to service the bulk of programming jobs which is for project with much longer lifetimes than engineer turnover rate. It the most expensive method in terms of both engineering and CPU time but the reduction in risk makes it worthwhile from a business standpoint. But this isn’t something to focus on in education or literature because it’s roughly the equivant to coding conventions. Anybody that can program can pick it up in approaching zero time.

What you just recounted above is the myth of the false promise of OOP. OOP has not reduced project failures. Attention to detail and rigorous application of such is what succeeds and sadly anybody picking up programming categorically lacks this and most never achieve it.

I can refer you to many other debates online, but kind of feel we don’t need to have one here on the matter… ;D

Err… Actually to stop spinning the wheels here on the rest of this particular post, so I’ll end my reply here… It seems we have differing opinions and continuing will dilute the larger topic.

Well yeah… and this

I keep forgetting. The argument “OO isn’t good enough so I use component based” is flawed. Component based is OO.

No one is saying it’s not OOP at least for Java, but by adding a CA API that is well defined and generic this unlocks implicit composition which is something stock Java / re traditional OOP doesn’t provide… It’s such a big game changer that it warrants being called COP (component oriented programming) versus OOP so folks don’t get it confused with inheritance / or explicit composition re traditional OOP.

[Catharsis]

@junkdog

What exactly do you mean by processing of entities in a “fully composed state”? In both artemis and ashley, entities are only a little more than glorified integers (entityId - and some additional metadata, not necessarily stored by the entity itself).

EntitySystem and EntityProcessingSystem which forms the basis for a lot of Artemis system usage from examples I’ve seen (the best being Naturally Selected 2D) iterate over entities sequentially and retrieves individual components to complete the logic at hand. Regardless of where the components are stored this general approach will have limits in speed especially considering the case where when one wants to broad phase process. In this case with Artemis each separate phase iterates over all entities multiple times while potentially needing to retrieve multiple components at each stage. The only thing that makes this a problem with Artemis / Ashley is that it’s baked into the architecture and is the enforced way of doing things. At least it looks inflexible from just reviewing the code.

I really do need to get more of an intuitive usage understanding of Artemis particularly your “odb” effort. I’m doing this right now actually in converting NS2D to Android / iOS builds for some demo work for a client while integrating their custom game controller (a touch screen on the back of a tablet). It’s kind of neat. Touch on the back controls the jet pack / movement and touch on the front anywhere controls the gun. I’m doing the modifications in the coming couple of days, so hopefully I’ll gain better visibility working with a good example, etc.

@junkdog

Or is the hint in your definition of “Type II Es”, being “fully generic” and having “self-modifying capabilities” (components as more than data-holders, I guess) - not entirely sure what either implies.

In my efforts the self-modifying behaviour is achieved by systems that are potentially activated when the CA API is invoked on a component manager. This allows entity management and classification to be done in a more dynamic fashion than baking these patterns into the ES API (type I). In the case of packing components for more efficient traversal what I’m aiming to do is when an entity is added to a manager the dynamic classification / packing system is run and stores the packed content in a different component of all combined & relevant entities added to the manager. When processing a frame a system can then just retrieve that one packed component and work over the packed data (ideally a primitive array). Remove the entity from the manager then it removes the tracking from the packed component and stores the packed data back into the original component.

You are brave with integrating byte code weaving and such. I’ve steered clear of that approach for developer setup / ease of use. The efficiency you are trying to achieve with PackedComponent is an interesting attempt. I’m just referencing this link (https://github.com/junkdog/artemis-odb/wiki/Packed-Weaver) for reference. The problem is not so much storing packed data, but making sure systems can iterate over the packed data and not make a bunch of method calls or component retrievals. This doesn’t seem possible currently with your efforts. I’d also generally be concerned about storing the packed data as a static array / ByteBuffer in the component class itself. This will not play nice with partitioning an ES.

@junkdog

https://github.com/junkdog/entity-system-benchmarks - Nothing complete, but it’s a start.

Thanks for doing this… I’ll take a look and maybe one day have some time to do some comparisons with my efforts.

[appel]

I spent a great deal of time profiling Artemis to ensure its design wasn’t bottlenecking the performance, and I did not find it lagging behind its OOP alternatives, e.g. dozens of thousands of entities.

If you have more than a dozens of thousands of entities, hundreds of thousands, then you’re doing something terribly wrong. No game should have that number of entities active in its world at a given time.

All I wanted was a easy design, where I wasn’t schizophrenically dependent upon some hierarchical OOP structure and was fast enough.

But that doesn’t mean Artemis-design is suited for all. It’s quite rigid when it comes to how entities are composed.

[Roquen]

You’re thinking about it the traditional way…

Yeap. And so are the majority of game industry leads & technical directors. It’s significantly less work, complexity and is significantly faster (for the statistically significant…otherwise who cares).

I think you may be complaining that historically the “best practices / design pattern”

No. I’m complaining that CS has a whole has stepped way from problem solving and into reusable massively complex systems and generations of programmer understand very little about programming in the pure sense. As an example around 2000ish I asked new grads from highly reputable universities questions like: “Visitor and command patterns are the same. What’s the mechanism?” I think one 1% could give an answer.

[junkdog]

@Catharsis

@junkdog

https://github.com/junkdog/entity-system-benchmarks - Nothing complete, but it’s a start.

Thanks for doing this… I’ll take a look and maybe one day have some time to do some comparisons with my efforts.

At work, so just a short notice: ashley 1.0.1 doesn’t compare favorably to artemis forks - it spends a lot of time sorting, but I’m assuming performance artemis vs ashley will converge with later releases. Briefly discussed on their issue tracker: https://github.com/libgdx/ashley/issues/36

[Catharsis]

@Roquen

You’re thinking about it the traditional way…

Yeap. And so are the majority of game industry leads & technical directors. It’s significantly less work, complexity and is significantly faster (for the statistically significant…otherwise who cares).

You are basing your arguments on conjecture at best. There are several high profile AAA game studios that are using CA techniques. There is no basis to the “significantly less work, complexity, and speed” comment. The CA approach is crossing the chasm in most new engine development with a reasonable percentage of game studios.

@Roquen

I think you may be complaining that historically the “best practices / design pattern”

No. I’m complaining that CS has a whole has stepped way from problem solving and into reusable massively complex systems and generations of programmer understand very little about programming in the pure sense. As an example around 2000ish I asked new grads from highly reputable universities questions like: “Visitor and command patterns are the same. What’s the mechanism?” I think one 1% could give an answer.

Your anecdotal evidence or point has no connection to CA or the original discussion and attempting to connect it to “best practices / design pattern” related thinking.

I can tell you this much though since I was finishing my CS degree around that time at a reasonably reputable university. Design patterns weren’t taught in the CS curriculum outside of perhaps a few professors making references here and there. Only those who had the motivation to learn outside the curriculum had a solid clue. There was no course dedicated to design patterns though I do agree that more or less undergrad programs at universities are not exactly graduating highly competent developers.

More or less I had to cut short our discussion because your replies are divergent from the original topics at hand.

[Catharsis]

@appel

I spent a great deal of time profiling Artemis to ensure its design wasn’t bottlenecking the performance, and I did not find it lagging behind its OOP alternatives, e.g. dozens of thousands of entities.

What you did with Artemis especially considering the experimental / trailblazing aspect is a complete success. For the type of games (2D / platform) created with Artemis it works well. I was very impressed when I checked out Naturally Selected 2D running on an iPad2 which is the lowest specced device I’ve tested with so far.

Any discussion I’m contributing to here is more or less aimed at floating ideas around future directions. Saying that Artemis is a really good “type I” ES does not detract from what was accomplished.

@appel

If you have more than a dozens of thousands of entities, hundreds of thousands, then you’re doing something terribly wrong. No game should have that number of entities active in its world at a given time.

All I wanted was a easy design, where I wasn’t schizophrenically dependent upon some hierarchical OOP structure and was fast enough.
But that doesn’t mean Artemis-design is suited for all. It’s quite rigid when it comes to how entities are composed.

I think you could say… No 2D game should have that number of entities active in most usual game play scenarios. All of the Artemis related games I got to compile surely fit those assumptions.

For a 3D game that is partitioning a larger world including partitioning the ES then entity counts could be in the hundreds of thousands (let’s say ~15-20k per region), but of course in most situations only updated for regions near the player(s). The design I’m working toward is meant to support a larger general use case, so the additional data oriented packing and efficiency will make a difference. I’d say a fair assumption about “type II” ES is that new architectures are less rigid and increase the throughput and conceivable data parallel aspects of the ES out of the box. In the case of my efforts integration into the larger engine architecture is much easier since the CA relationship is inverted with the CA being a superset of the ES.

[Roquen]

We’re mostly going in circles.

There are several high profile AAA game studios that are using CA techniques.
…
The CA approach is crossing the chasm in most new engine development with a reasonable percentage of game studios.

Provide some references. Easy to find avoid OO use PODs and data-oriented…but show me some CA.

There is no basis to the “significantly less work, complexity, and speed” comment.

Of course there is. What a proper component based system must do requires the complexity. Writing the thing or even using a library for virtually all use-cases is more work then writing the specific case (except if your writing in a grab-bag composition language, but then you don’t need CA). General purpose is always less efficient than specific case.

…has no connection to CA or the original discussion and attempting to connect it to “best practices / design pattern” related thinking.

But it does because that’s how it’s virtually always written about and how people tend to use it. You’ll notice I gave no comment on delt0r’s post…if you have a reasonable use-case then why not.

… your replies are divergent from the original topics at hand

Which was Orangy Tang attempt to convert a simple (with alot of devil in the details) problem into a very complex problem with still the same devil in the details.

[princec]

So… can anybody tell me what’s wrong with SQL to do all this, better, than Java?

Cas

[Riven]

The SQL language could be great, but all the engines backing the language focus on (and are optimized for) a different use case.

[princec]

Does it actually need any particular use case? Looking at the classes of “problem” solved by CAs and entity systems here what I see is just basically set operations on relational data. If one were dead set on having it in a JVM there’s HSQLDB (which IME was actually bloody good - shame it’s got so many showstopping bugs in a multiuser environment…)

Cas

[ewg]

I think the problem with SQL is that there is no way to specify inside the SQL syntax itself the type of data structure or lookup algorithm to use for your queries. All of that is left as an database implementation detail. So if you want decent performance and to make sure your queries do not end up looking over you entire dataset (or creating huge temp tables) instead of doing fast lookups you end up having to use the features outside of the sql language itself that are specific to whatever database you are using. You will notice if you look for SQL help online there is no place that will give you advise on query optimization without specifying which database you are running on first. If you are thinking of creating a sql like language I think you should just build the ability to specify the specific data structure and algorithm to use inside the language itself.

For CA I would love to see a comparison of competing implementations written in this style to code written without CA based on both speed and code elegance. Would it be a fair comparison to do a non-CA based implementation of junkdog’s benchmark? I have not looked into the details yet.

[princec]

The indexing and query systems of RDBMSes are pretty advanced… the point of using an RDBMS to do this is precisely so you’re not bogged down with pesky implementation details (which is the whole issue that plagues any Java implementations of this idea) and instead concentrate on what you’re trying to achieve rather than how to achieve it (and worse, bullying something decidedely resistant into doing it for you).

Cas

[Catharsis]

What Riven said of course… and…

@princec

Does it actually need any particular use case?

Performance would be one. I mentioned other reasons / answers to your question prior why I’d prefer the CA / query mechanism to be Java code. Some of them:

• Runs anywhere w/ no dependencies and small code footprint
• Data is stored naturally in the format most useful to send to the hardware at hand.
• Logic is not far removed from the data and directly acts upon it
• All code and data can be versioned, modularized, and deployed with one tool chain.
• Refactoring support is super strong for pure Java.
• Cross-compile to iOS via RoboVM (been really impressed w/ LibGDX in this regard lately!)

@princec

Looking at the classes of “problem” solved by CAs and entity systems here what I see is just basically set operations on relational data.

That is an aspect to it all, but one of many concerns. The runtime aspects of being able to store components by extensible enum and use that same enum as data defining a category implicitly for event distribution in the EventBus is super powerful use case.

I’d be down to see someone experiment in that direction, but not for me… ;D