Efficient caching for Entity System

Newbie & Debugging Questions
#1

I am looking for a way of doing an efficient caching system for an ES.

all update are made in memory (but only requiered/active parts of the DB are loaded) and periodically the cache is saved to the DB, the problem I found is that how to efficiently cache components of my entities without explode memory and keep an extremly fast acces to component based on their entity ID ?
for now I got something like the following :

use native array for memory storing
=> String componentLabel[];
=> Point3D componentLocation[];

and use an hastable to get entity index in natives arrays :
=> Hashtable<Integer,Integer> entityIDtoNativeArrayIndex;

I only use one hastable that give me the index of the entity for each possible components, I was wondering if anybody have a better idea ? one problem I could face later will be that :

  • memory used for an entity with only one component will be always equals to the size of all componenets.
  • a new componenent will make a big memory grow (for all active entity it will exist an entry).

any (simple…) idea to compact this structure (wich is basically a 2D array with size equals to activeEntity*nbComponent) in an efficient manner ?

EDIT:

what I need is to be able to get a list of all entity having a component of type “XXX” while still being able to faslty access some other component of those entities.

something that may look like :


 havingPhysics[]=getListOfHavingPhysics();
 for(int n=0;n<havingPhysics.length;n++)
 {
	applyPhysics(havingPhysics[n]);
 }
 
 havingAppearance[]=getListOfHavingAppearance();
 for(int n=0;n<havingAppearance.length;n++)
 {
	applyAppearance(havingAppearance[n]);
 }

where havingAppearance & havingPhysics may overlap

#2

I got an idea that sound pretty good for me (to be honest I am still in R&D for that…)

maybe I could have an object Entity that will have an integer for every component, this integer will be the native array index
like “grosso modo” :


Entity
{
 int haveApperance;
 int havePhysics;
 int haveLocation
}

with -1 means (null) => no component and other value equals to the index of the component in the native array, but this will be still 4 bytes per existing componenets for any existing entity this does not completly solve the memory probleme but seems to solve the fast “cross-component” access

#3

first :

an entity classe :


Entity
{
 int haveApperance;
 int havePhysics;
 int haveLocation
}

and an entity native array :


Entity entities[];	//an array with all loaded entity
Hashtable<Integer,Integer> entityIndexById; //Enable to get an entity index in the native entity array with its ID

some components natives array :


CMPPoint3D location[];
CMPAppearance appearance[];
CMPString label[];
etc...

with each component having a properties entityIndex on a supperclass


SuperClassComponent
{
 int entityIndex;	//index of the entity in the native entity array
}

ClassComponentApperance extends SuperClassComponent
{
	//entityIndex is inherited
	int color; // ....or any other apperance component properties

}

so I can get all entity with a given component by just reading the component native array, and access any other component faslty


for(int n=0;n<appearance.length;n++)
{
	applyApperance(appearance[n]);
	int entityIndex=apperance[n].entityIndex;
	Entity e=entities[entityIndex];
	location=location[e.locationIndex];
}

sounds pretty efficient, the only thing more I would requiere is an Entity class that dont always use an int per possible components…

EDIT: typos

#4

I personally tend to optimise entity management in each program i make. I find that I always do it different.
I am curious to know, what sort of game/application you were developing this for?

#5

so finally end with something that I am satisfy with, will not explain details as it is internally very similar to the previous post :

one native array for all components :


CMPLabel labels[];
CMPAppereance apperances[];
etc....

an entity array and a fast index acces Hastable


Entity entities[];
Hastable <Integer,Integer> entityNativeIndex;

the new point is that I added an EntityManager that is responsible of adding new entity and/or components, every component added to an entity is added by the entitymanager every new entity is added by the entity manager, this enable good integrity and nivce looking code and good eficiency,
code looks like :


EntityManager em=...;
//load from DB
em.loadEntityFromDB(102);
//or build by hand
em.addEntity(103);
CMPLabel l=em.addComponentLabel(103);
l.value="label103";
CMPColor c=em.addCMPColor(103);
c.value=0xFFFFFF;

than later


CMPVisible visibles=em.getVisibles();

and still able to threat componnents as blocks for systems or DB saving with cross access to other components of the current entity


for(int n=0;n<visibles.length;n++)
{
 doVisibleSomething(visibles[n]);
 //and keep direct access to any other component
 Entity e=visibles[n].getEntity();
 e.getAnotherComponent().value=3;
}

the EntityManager is responsible to maintain coherence between component & entity, all storage (component/entity) is made in native arrays, the Entity class is only usefull to enable cross component access and fast entity loading (in DB it give for an entity all components it use => direct DB modifcation are then inded not alowed to keep DB integrity).

I understand what you mean and have similar feeling than you, it would have been a lot easier (faster) to make something more specific but I really wanted to find something more generic to let ‘more open ways for the futur’

this is for an online game : MMO and we can say in a certain manner MMORPG (even if there wont be any magic, potion, dwarf or such…)

EDIT: the slower part is to get an entity by its ID (use of the hashtable) but it should not be used a lot, all other acces are direct acces : all component keep a direct link to its entity and all entity keep a direct link to each of its components

#6

Try the hasmap with cuckoo hashing someone here made.

#7

thanks as I said above it is not a bottlneck but if you can point me a link to this one and if it make a significant improvment it will be nice anyway