Vote for Structs!

Miscellaneous Topics
#1

Programming games in Java … is a bit like making spaceships out of Lego.

Just one of those strange little analogies that pops into one’s head now again. Remember what making spaceships out of Lego was like in the mid-70s? All your spaceships were rather square, and opaque, and didn’t have any lasers or antennae. But the wise and benevolent Lego company realised that we needed some special-purpose bricks to make our spaceships especially sleek. So we got lasers, and little spacemen with oxygen tanks and helmets, and ariels, and thrusters, and wing-shaped bits, and computer panels, and transparent screens…

Right now I feel like I’m using Lego in the early 70s when writing in Java! But if you all go and [b][i]vote for structs[/b][/i] we’ll have a rocket thruster piece to play with! It’ll still be square and laserless but we can get those bits for Christmas! Vote vote vote! Even if you wouldn’t know a struct from a fruitcake you’ll be helping us all!

Cas :slight_smile:

#2

[bad taste]
Lego never made any special purpose heat shield bricks though, maybe that was NASAs downfall ::slight_smile:
[/bad taste]

#3

you got my vote.

i browsed spgl source’s and found StructBuffer is that you want to include in jvm or is that a diffrent thing?

#4

Nah, structbuffer was just some half-arsed attempt by me to wrap things up a little but I’ve never used it.

Cas :slight_smile:

#5

I have just finished reading the RFE you posted on Structs. I similiarly was pushing for a “struct” data object in Java last year.

However, the mechanism you suggest I strongly disagree with (at least as much as I understoof it from the RFE).

Specifically, requiring the struct to have a buffer backend. This effective makes a single struct instance even larger in memory, and one of the benefits of a struct should be less memory than a equivalent Java class. (IMHO)

What you are describing is a bytebuffer wrapping mechanism. “Real” java structs do not (should not) be anywhere but in Java memory space. One of the HUGE gain for have structs in Java is much better memory use for large arrays and other collections of struct objects (i.e. Matrices). In addition, Java structs shoudl be methodless and non-inheritable - i.e. c-like struct, for absolute minimum memory footprint and maxium performance. The closest thing to this now in Java is simple arrays of primitives, but that is a very non-OO way to code up things and is very ugly syntax. (currently how we do 3D character animation)

Please give more comments, I would love to hammer out a complete view of this stuff publicly now. (although you on this list are damn quick, much quickly than I get to respond it seems!)

#6

I too have been hammering on about structs in Java since the boards started over 2 years ago but it was a bit like arguing with a brick wall :-/

The mechanism I proposed is therefore a cunning twist on the simple struct-as-primitive-type idea. It’s cunning because it fits in really nicely with NIO, which is where I see structs being most useful (for me, anyway).

There is indeed an overhead with using an object to represent a single struct but I’ve cunningly designed the mechanism to re-use instances. And indeed without a reference type it’s going to be pretty inefficient passing structs about to Java methods.

Also, I can’t really see anyone accepting that a struct can’t have methods associated with it.

So you see, what I’ve got here is a compromise which is likely to solve most of the problems we’ve got and fit in nicely with what we already have.

You could still, for example, have an exceedingly space efficient BSP tree implemented as structs. Pointers to nodes are simple stored as int buffer offsets; you only need 1 instance of BSPNode to walk the tree. It takes up basically the same memory as in C. Nice or what?

Vote vote vote!

Cas :slight_smile:

#7

No offense but if it’s a cunning twist on structs, maybe it’s not “structs”

I see no reason why the structs I explained are passable by reference, in fact, they must be just like objects.
I’m not sure I understand how you re-use instances…

I disagree. That is what a struct is, a compact, non-inheritable, methodless object. Structs in C do not have object methods, they are functions that can operate on those structs. In OO that would be in a different class. The typical example is a Vector struct, and Vector utils class that processes Vector structs.

Well if it is a compromise, then I suggest calling it something different. A struct has very specific meaning and more importantly expectations for the user of them.

What you have described as I understand it is some native access improvement. Structs can and should be a pure Java concept, and are agnostic to native access.

#8

You got my vote princec.

#9

Well Shawn, what it seems you’re saying is that I simply shouldn’t call it “structs” because it clashes with an old, old, idea from an ancient language called C…?

I want to have my cake and eat it!

Cas :slight_smile:

#10

In all seriuosness, the idea of a struct ( a compact, non-inheritable, methodless object ) is a great addition to the Java Language, that I would like to see.
This is not what you are describing.

Now that “old, old, idea from an ancient language called C” is still in existence, and more importantly here, in heavy use in gaming libraries, for example RenderWare, the biggest money producing game engine out today.
Besides is existing use, it is a good technique (dare I say “best”) to use for many, many problems. But, unfortunately, what you are describing is something different. You have to have some respect for your fellow programmers, and except the working definition of “Struct”. You can’t simply reinvent what that means and expect it to be accepted by the existing community.

#11

Well, can we call it a ‘mapped object’ then and will it then be a great idea :slight_smile: ?

Cas :slight_smile:

#12

Sure! :slight_smile:

Perhaps add native to that name somehow, althought “Native Mapped Object” seems long.

Maybe there is some ByteBuffer name derivative that would sound better(sometimes names are that critical to acceptable and avoiding confusion).

Or another possibility would be a Struct type, and then a NativeStruct can mapped directly to native buffers. But here there still would be no methods or inheritance, like the C-sde buffer and C-struct. I hate to have yet another layer for the MappedObjects,i.e MappedObjects layer NativeStructs that wrap ByteBuffers. This would maintain “struct” functionality independant of native buffers AND independant of java objects, as well as allow your MappedObject idea, but it certainly seems like too much.

How can they be seperate and both have the mapped native access you want, where on is a real Java object and the other is the c-struct-like objectless type?

Ideas anyone? :slight_smile:

#13

I still think Struct is probably the most accurate name for this feature. After all, the notion of a Java array is almost completely different from the C version and no-one’s complained. Much.

I spent long and hard trying to figure out how you could actually implement pure C structs in the Java language and concluded there would be some problems.

To implement a struct you have to make a decision on how it behaves. Either it behaves like a primitive type, or it behaves like a reference type.

If it’s a primitive type, we have to pass-by-value to method calls, which will involve possibly large amounts of data copying. Furthermore you are then restricted in that you cannot have circular struct references in definitions as there is no pointer type in Java. So there goes BSPs, linked lists, quadtrees, etc.

If it’s a reference type then it will need to descend from Object and be garbage collected like anything else, so the whole memory advantage is blown again. However it then fits seamlessly into Java’s method dispatching system etc. and will allow it to have methods associated with it. Again, I can’t see a good reason not to have methods associated with them - it’s one of the best bits about OO programming, and there’s no reason to throw the baby out with the bathwater. And finally of course if you’ve got a native buffer filled with native C-structs you still can’t manipulate that buffer directly, it needs to be read into objects and copied back again.

I could in fact see only one way which we could get the memory efficiency of a C-struct, the semantics of a Java Object, and very close to the memory bandwidth efficiency of using C-structs, without breaking the VM too much.

That’s to restrict the use of structs to storage in ByteBuffers - think about it, it’s the only way to avoid the Object overhead - and map their fields using JVM trickery direct to the memory of the buffer. A struct effectively becomes a pointer to a bit of memory in the buffer - just like in C - but it’s a proper Java Object so we’ll call it a Struct and not a struct.

This means that a Struct can’t contain Object derived fields, only primitives - as there is no semantic for mapping arbitraty Objects to bytes in a buffer! - but it would be reasonable to allow other Structs inside a Struct. What you still can’t do is store a pointer to another Struct - but instead you’d use an int offset into the buffer (measured in Struct sizes, not bytes). The whole mechanism is so beautifully simple and elegant, but above all, it’s super-efficient, totally safe, and easy to implement. I’m quite pleased with myself for thinking of it :slight_smile:

There really isn’t any place a natural C-struct fits in the Java language or the VM, you see. To get the RFE through the hurdles it’s got to appeal the VM engineers and language nazis, and they will never buy the idea of implementing C-structs in Java because of the aforementioned wrangles.

Cas :slight_smile:

#14

No vote from me. 8)

Stucts in Java is against Java’s nature. Objects should do in all cases. If you need something faster/more-flexible/more-reliable you should be using c/c++ by now (and don’t you dare flaming me just because I’m right). Pushing to its limits a language who’s programs are interpreted, makes no sense at all.

Plus, Sun doesn’t care about the language at all. Why should we? :stuck_out_tongue:

PS: Hey I’ve got an idea! Instead of stucts let’s add assembly support! That would speed up things even more, wouldn’t it? ;D

#15

" If it’s a reference type then it will need to descend from Object and be garbage collected like anything else, so the whole memory advantage is blown again"

I think you are missing the memory advantage I want. I don’t care that it is GCed, in fact, I 'm counting on it.

I don’t care about Native memory ( I do, but for agrument sake, I don’t) so why should be Struct have anything to do with ByteBuffers. Now if the VM wants to map it that way behind the scenes, I could careless.

What I want/need, is a c-struct Struct, simple, clean, efficient.

Yes, you can build trees with structs, they can have pointers, so Java Structs can have object references.

Yes, they should be passed by reference and not by copy (this one is obvious - this isn’t LISP :-))

No, they do not need methods or inheritances, and that IS the point of the object I am describing. Structs hold data (primitives and references) and nothing else. If mapping them to buffers limits their references use, forget it. I think about it, that will never get pass the VM/Language people - a new object-liek type that can’t reference the existing Java objects except through user implemented indirection tables!? Whoa :slight_smile:

The idea here is to make a Java object type that would work like Vectors and Martices do in C, because it is the lowest possible memory overhead and run-time overhead, and for certain programming techniques is cleaner than straight OO (ok last one is opinion :-))

The thing you are desrcibing now has too many exceptions (no Java references) and misses the run-time gains (no method reflection overhead, JIT, etc.)

The computation math folks have been discussing similiar contructs since Java ran on vector processors, because the overhead when you have 100,000s of vectors and matrices (also in 3D) ends up blowing your memory and run-time access performance.

One more wrench I want to toss in there since we are “dreaming” - I want a way I can declare arrays of your structs or my structs created in such a way that they are contiguous memory, like in C/C++, that way the caching behavior of sequential access is much better :slight_smile: (since that is in high use in the struct techinques involving vectors etc - although I imagine a VM can do this behind the scenes, at least for “fresh” arrays)

Are we any closer? :wink:

#16

[quote]No vote from me. 8)

Stucts in Java is against Java’s nature. Objects should do in all cases. If you need something faster/more-flexible/more-reliable you should be using c/c++ by now (and don’t you dare flaming me just because I’m right). Pushing to its limits a language who’s programs are interpreted, makes no sense at all.

Plus, Sun doesn’t care about the language at all. Why should we? :stuck_out_tongue:

PS: Hey I’ve got an idea! Instead of stucts let’s add assembly support! That would speed up things even more, wouldn’t it? ;D
[/quote]
One, Java does have assemble access right now, through the JNI. We will be using it to access P4 SIMD vector instructions for batch 3D stuff.

Two, Java is NOT interpreted on all platforms, namely J2SE, it’s JITed so the most running parts are native executible.

Three, there is not reason that “Struct” type is not an Object as well, it just a Data Object, not unlike the Java Data Object API, or any relational databases use.

And you can’t really say if you want something else go somewhere else, because java is evolving based on user and industry input, and it BETTER evolve, or .Net languages will over take it. C# allows very similiar features to what we are saying here, and it is the fastest growing language right now - [source Slashdot links]

Here, we are just trying to get clear a solution for significant problems we face using Java.

#17

One last note on the “If you need something faster/more-flexible/more-reliable you should be using c/c++”

J2ME-MIDP does not do reflection or floating point math.
Java itself has variants, so there is no reason it can not be added to, or in that case subracted from.

There is precident for changing Java through user requests.

#18

Hi :slight_smile:

Hey take it easy big guy! First off, I know that we have m/c support through JNI (ain’t that beautiful? ;D )

As I was saying, if you read carefully what I wrote, you’ll see that I’m 100% right. Java was invented for a reason. You choose it for what it is.

There are other alternatives, not too different, that offer the things you want. Java cannot be a ‘one size fits all’ language.

That would defeat its own purpose.

Cheers.

#19

[quote]Hi :slight_smile:

Hey take it easy big guy! First off, I know that we have m/c support through JNI (ain’t that beautiful? ;D )

As I was saying, if you read carefully what I wrote, you’ll see that I’m 100% right. Java was invented for a reason. You choose it for what it is.

There are other alternatives, not too different, that offer the things you want. Java cannot be a ‘one size fits all’ language.

That would defeat its own purpose.

Cheers.
[/quote]
Didn’t mean to come off - loud? :slight_smile:
I’m confused about what you are 100% right about?

Also, what reason was Java invented for again? And in any case, should the reason it was invented, limit it? That doesn’t seem like a very strong agrument to me.

#20

It’s no argument at all. I don’t think cmr here has ever tried to do anything like what we do in Java. We know it can be done but it’s a pain in the arse right now, and 50% of the speed it could be. I merely want to make it easier, and faster, without a) making life hard for the VM engineers and b) scaring off newbies. Only people who know why Structs are necessary need to use Structs. This ain’t like dustin’ crops, boy.

Besides, the solution I’m suggesting is incredibly simple and not a massive change to the JVM.

Shawn’s suggestion is rather more complex and I think you’ve missed some critical timesavers in your idea. We’ve got a neat way to get contiguous areas of RAM to store structs in already - bytebuffers!

And also, there’s no way you can have references to structs in Java, and hope to get them garbage collected, without the same overhead as a normal Object. It would simply be far too difficult to figure out, let alone implement; but deriving Struct from Object is a no-brainer. Just point an instance at an address in a buffer and bingo - all the properties of both an Object and a C struct. The most useful advantage of all this - as someone who’s really pushing the boundaries of Java here, for sure - is that I get access to native side rendering structures for free as well!. With pure Java native structs, you’d still be forced to write them into a buffer after you’d fiddled with them, or devise something that looked uncannily like my Struct implementation over StructBuffers.

Cas :slight_smile: