JGLOOm - Loading Every File Format for Every OpenGL Lib*

Thanks! We’d love your help. Here’s some pointers:
Currently the image decoders read from an interface called [icode]Resource[/icode], we can easily make a new [icode]GWTResource[/icode] that could be anything you’d like, and the input stream could return null. This would pair with an [icode]GWTImageDecoder[/icode].

gouessej mentioned we are using AWT classes in the core library, and I am rightfully ashamed of myself. I’m working on making a new class that contains the basic data for uploading to a texture, this way we can abstract out all that nasty AWT stuff.

I’ll take a look into this for a few days (had training programme in college, so I can’t immediately work on this). And here are some things to keep in mind when working with GWT, and IO in specific.

[] There is no synchronous IO. IO should happen in form of XmlHttpRequests and we should use callbacks to do the result.
[
] GWT has no support for threading, and waiting on the main thread will cause the browser to become unresponsive.
[] GWT doesn’t support reflection because it is a source-to-source translator.
[
] There are only a limited number of classes emulated from the JRE, no NIO classes are available.

These are the things to keep in mind, and I think you are violating all these right now. Even though you get the data of the files through XHR, you can’t directly get the pixel data out of it. The only way to load images is to inject a IMG tag to the browser and set it to not visible, and once it’s loaded, you have to pass it directly to WebGL.

We can of course work-around the NIO classes by implementing a GWT support library with super source. Hope we can deal with these.

Ecumene, your third link is broken.

Sorry for the dummy question but what is your target? Which developers should use your library?

Actually, you’ll need at least a math library and you’ll need to provide something to manage the hierarchies, this is already implemented in numerous typical scenegraph APIs, middle-level and high-level frameworks and toolkits. Who would choose your library rather than LibGDX 3D API, JMonkeyEngine 3, Xith3D, Java3D, Unlicense Lib, JogAmp’s Ardor3D Continuation, …? Moreover, lots of those APIs focus on a few file formats (Blender for JMonkeyEngine 3, Collada for the legacy Ardor3D, FBX for LibGDX) because it’s better to support fewer formats very well than lots of format badly and there are numerous tools to convert 3D models. Some formats are better for long time storage and interchange but bad for use at runtime in a game, some formats are only good to be used with a particular editor and shouldn’t be used for interchange or in games. If you support all 3D file formats, you’ll end up with an API as complicated as those already available now, is it your aim?

Finally, I advise you to take some time to wisely choose a solution to support WebGL. GWT isn’t the only available option. Bck2Brwsr doesn’t have a WebGL binding yet but it supports the 2D canvas, it supports a larger subset of the standard JavaSE API than GWT and it’s faster (3 to 10 times slower than Hotspot). TeaVM supports WebGL.

Cough take a good look at assimp; oops already mentioned this… ::slight_smile:

Khronos Showcases Significant glTF Momentum for Efficient Transmission of 3D Scenes and Models

Oops. Already mentioned that too… :persecutioncomplex: glTF is built for WebGL in mind and then some!

On glTF (don’t just take my word):
“The world has long needed an efficient, usable standard for 3D scenes that sits at the level of common image, audio, video, and text formats. Not an authoring format, or necessarily a format you would use for a hyper optimized platform specific application, but something at home on the internet, capable of being directly created and consumed by many different applications,” said John Carmack, CTO of Oculus.

My take is first and foremost provide a knock out glTF implementation 100% and adopt / support any translation efforts (FBX-glTF / FBX2glTF whitepaper / Collada2glTF) that are available. Don’t try to solve the all formats issue from scratch. If that really is an itch fully support assimp as a side show, but otherwise create the best glTF importer you can and you’ll be ahead of the game. Take cues from the partial assimp glTF import / exporter and examine the major javascript 3D frameworks that have adopted the format (three.js / babylon.js) and consider carefully a Java based format / essential rendering routines if not assimp derived.

Heck I’m tempted to take off a week from what I’m working on now and knock out a solid solution (err, of course based on TyphonRT… ;P… sigh… open sourcing plenty of things but not this:()…

LibGDX, JMonkeyEngine, and all the others are mainly game-engines, there’s plenty of use for high-preformance java graphics outside of games (computer simulation is what I made it for, but it can also be used in, for example, data-visualization) JGLOOm is fairly lightweight compaired to the other graphics solutions.

There’s good reasons to choose JOGL over LWJGL in some cases, and WebGL over LWJGL (of course). There should be an easy and similar-code-wise method for loading models for them

Good point, we’ll use jAssimp’s model importing standard, convert it to pure java (no NIO) and support a few of the masses choosing.
My thoughts on which formats, with good reason I’ll add more:

  • OBJ
  • FBX
  • MD5
  • FBX
  • glTF

Zach and I are carefully working out the kinks over the phone, GWT is a very sensitive toolkit. We have a solution (somewhat) to every problem that may persist, allowing users to be able to decode any format similar to how you would in the LWJGL/JOGL ports. There will be options to support TeaVM, Bck2Brwsr, as well as GWT. The jgloom-core module will have to have as little code as possible, just simple stuff to decode file formats.

Yo, you can talk the talk… But can you walk the walk?
It’s just JSON, it’s very easy to serialize to java objects. As long as JSON serializer works in GWT, I’m happy. I can add you as a contributer, when we finish the jAssimp - GWT friendly we can get to work. (If your tempted :persecutioncomplex:)

While supporting a few file formats very well, there is only so much you CAN support. Meaning if you handle everything like its supposed to 100% of the time, or you are happy, don’t be afraid of starting work on something else if you so wish :slight_smile:

[s]Update on GLTF, GWT can’t support it. I’m calling it now!

Unfortunately it points to binary buffers for mesh information,

  1. GWT is a source-to-source compiler meaning we can’t use native code, or binary formats (unless you load it in booleans or something)
  2. WebGL4J doesn’t have any methods to upload native-style buffers anyways, so it’s a bust.

Unless we modify the format to read from a plain text version of the meshes as simple (xyz,uv…) stuff, GLTF format won’t work. Thanks for the recommendation though, Catharsis!

EDIT:
Update on FBX, GWT can’t support it either.

Being a binary format, Javascript doesn’t have any way to load it. Leaving MD5 files as the only candidate for skeletal animations, unless someone else has anouther plain-text animated-3d-model solution [/s]

I just found an FBX loader in javascript as well as documentation for using glTF with WebGL. Herp derp:


http://threejs.org/examples/webgl_loader_fbx.html

Half of the scenegraph APIs I quoted aren’t mainly for games. If the main “advantage” of your library is its light weight, you’ll aim a tiny niche containing those who need more than just a set of bindings for OpenGL & OpenGL ES APIs and those who need less than a middle-level framework like LibGDX. This niche is already occupied by jReality and jzy3D. You talk about computer simulation and data visualization but I don’t see numerous formats used in scientific visualization in your lists and unfortunately, as time goes by, there are fewer of them supported in major 3D editors like Blender (especially since its version 2.50). PLY and STL models can be converted into OBJ, it shouldn’t be a problem. I see neither U3D nor Collada in your list.

What formats are used?

By computer simulation, I meant primarily on the web. Hence the webGL

GWT and JS both have the support for reading binary files too. You can create an XHR to a file, set it’s type to ArrayBuffer and receive an array buffer, which is like a container for binary data. You can create a DataView on top of it which behaves like a DirectByteBuffer. For example, here is a GwtDirectBuffer class that I wrote for the GWT backend of SilenceEngine.


public void readBinaryFile(FilePath file, UniCallback<DirectBuffer> onComplete)
{
    // Create a XMLHttpRequest to load the file into a direct buffer
    XMLHttpRequest request = XMLHttpRequest.create();
    request.open("GET", file.getAbsolutePath());

    // Set to read as ArrayBuffer and attach a handler
    request.setResponseType(XMLHttpRequest.ResponseType.ArrayBuffer);
    request.setOnReadyStateChange(xhr -> {
        if (request.getReadyState() == XMLHttpRequest.DONE)
            // Invoke the onComplete handler
            onComplete.invoke(new GwtDirectBuffer(request.getResponseArrayBuffer()));
    });

    // Send the request
    request.send();
}

And now I have this method to read the file as a binary file. Note that this is callback based because of JavaScript’s single threaded nature. Once done, the callback is invoked, and then you can simply read the bytes from the buffer as you wish.

:o ???

JSON description -> binary blobs + assets -> NIO -> profit…
|
±> shaders -> profit…
|
+ scenegraph -> pure java / component architecture preferably -> profit…

Hey man, I can guide the path, but shine you must… :stuck_out_tongue:

Currently walking the walk plenty busy creating fundamental tools; that one should be over 3k daily downloads by end of Sept… rounding out the arsenal so to speak… more to come…

NIO Buffers (and all the other natives), to my understanding, don’t work with GWT, right?

That’s the part that scared me, but SHC showed you can still use binary formats in GWT!

Typhoon looks very interesting, I knew you had your Java app but you seem to exploded in crazy works. Nice job!

Thanks for the tips on importing glTF, hopefully that’ll be the first format we support.

PLY, OFF, STL, Mathematica Graphics 3D, VTK, …

Actually, if you separated the geometry storages used during the parsing (pure Java, no NIO at this stage) from the creation of meshes and nodes containing the data for OpenGL (typically the NIO buffers) and if you chose a math library widely used by several engines (for example Vecmath, used as is in Java3D, a variant used in Xith3D, another variant used in JogAmp’s Ardor3D Continuation), those engines could use your stuff, maybe with a Maven plugin to adapt the imports, and you could use most of their loaders. However, it’s up to you and I understand that you prefer using jAssimp.

My plan was to have an AiSceneRenderer that simply renders a scene given an AiScene. Then we’d have different variations of LWJGLAiRenderer, WebGLGWTAiRenderer, and JOGLAiRenderer. Since there’s no universal model class we can easily support lots of formats and loading libraries.

I’m concerned now about Vecmath libraries, as a lot of those libraries use a proprietary vecmath solution…

My advice is to create an interface with OpenGL functions that the renderer would need, and implement the renderer in the core. The interface with the opengl functions will be passed to the renderer from the backends.

Proprietary? It’s under GPL v2 with Classpath exception (i.e non viral):
https://github.com/gouessej/vecmath/blob/master/LICENSE.txt

What did you mean exactly?

Yep ArrayBuffer / and whatever view matches for the copy.

TyphonRT has 13 years of crazy works behind it though just not open source for all to see; so as far as crazy works what’s visible is the tip of the iceberg… :wink: I just want to get to something commercially launched that justifies any ROI before open sourcing TyphonRT. TyphonJS I’m open sourcing and that has been refreshing to get things out for folks to use, but haven’t publicized anything yet about it. TyphonJS is also a test on building the infrastructure for the many-repo approach. Even a year ago that approach only is possible on Github financially speaking for open source repos (free). I believe ~6 months ago Github finally changed the pricing structure for unlimited private repos per org / user for a flat fee per user. Even then that doesn’t fit the model I’m developing as I treat Github organizations as a “component category” hence if I want private repos across ~30 organizations for one user that is ~$210 a month; better than it used to be though. There is Gitlab (free private repos), but I’m going to evaluate that later. TyphonJS is spread over ~25 organizations currently each with repos specific to the category that the organization represents. Some tooling is available (how the listing on typhonjs.io is created), but I’ve got a few unpublished tools that given a regex and Github token for instance all repos of TyphonJS across all organizations can be cloned in bulk and WebStorm projects automatically created and npm / jspm install run in one batch (no need to manually install everything which would be crazy)… Eventually a GUI configuration tool for apps will allow appropriate selections for an end project to be created with referenced NPM / JSPM modules that create the appropriate project with all resources, etc… Lot’s more, but I’ll stop here though as this is a JGloom thread… :stuck_out_tongue_winking_eye:

Also keep in mind the glTF binary extension

So the canary in the coal mine in all the discussion thus far is that what you are trying to pull off is complex. If you go down the traditional OO route you’re going to be screwed if not more so than a generic OO entity system. Especially screwed if you ever want to support Vulkan efficiently.

The direction I recommend is a purely event driven path for model loading and rendering. There are no direct connections between subsystems. Just events that each subsystem responds to and posts further output events handled by the next system in the chain. This works for loading and rendering or whatever else JGloom does. Unfortunately there is no publicly available efficient / component oriented event bus for Java out there (yep, TyphonRT is based on this).

Let’s assume a JGloom instance manages all models / scenes loaded. JGloom has an event bus internally used.

Promise modelID = jgloom.load(a loaded gltf file);

Let’s assume “load” might introspect the type of file or data being loaded. A modelID is assigned, promise created and returned after posting two events on the internal JGloom event bus. The first, CREATE_MODEL, forwards on the model ID and promise which is received by a management system. The second, LOAD_GLTF, with the raw file data after introspection and associated model ID.

The GLTF parser receives the LOAD_GLTF message and then starts unpacking and finds out there are 5 things to load. First it fires an event on the event bus PARSING_GLTF with the model ID and how many assets are being loaded which is received by a collector system which creates the complete model instance / component manager (hopefully). The parser then fires off 5 events (let’s say there is 1 texture, 2 shaders, 2 binary blobs), so the following is punted to the event bus with model ID and raw data from the glTF file: LOAD_GLTF_TEXTURE (x1), LOAD_GLTF_SHADER (x2), LOAD_GLTF_BINARY (x2). One or more separate loading systems receive these events and then create the proper resources for the GL or Vulkan or whatever environment being used. IE when you create a JGloom runtime for GWT you load GWT loader / renderer systems which for instance store binary data in ArrayBuffers or JOGL, LWJGL with NIO buffers, etc. Each of those loader systems create the proper assets / format and post with the model ID: LOADED_GLTF_TEXTURE (x1), LOADED_GLTF_SHADER (x2), LOADED_GLTF_BINARY (x2). The previously mentioned collector system receives these events and adds the loaded data to the model and once all assets are received given the asset count emits a LOADED_MODEL event with the model ID and model which is picked up by the management system tracking the initially returned promise replacing the promise placeholder with the actual managed model then completes / fulfills the promise. I guess you could also get fancy and support reactive events (RxJava, etc.) or expose an external eventbus as well as promises.

The user can then retrieve the actual model from JGloom… Let’s say jgloom.get(modelID).

Another great reason for this kind of architecture is that it’s really easy to provide a JGloom implementation that entirely excludes any renderer and defers to the app to render or even a model creator system that loads assets into an existing engine structure if applicable such as JMonkeyEngine, etc. The latter of course if the engine defines a fine grain API to import assets / scenegraph nodes, etc.

This is still all a strawman architecture idea, but perhaps gives a different perspective. The renderer would a bit more complex… Considering batching, animation, culling, etc. etc. At a certain point the user needs to be in control of these aspects. The nice thing though is that a fully decoupled system could load a user provided system without the complexity of a complicated OO dependency hierarchy.

While not super deep on details check out Dan Baker’s / Oxide Games talk on Vulkan and Ashes of Singularity engine. Take note when he mentions that the renderer can be swapped out and the rest of the engine / architecture doesn’t care.

That’s far above the task of a loading meshes library :stuck_out_tongue:

Ya, that’s fair. We can probably just generate AIScenes to buffers compatible to each library only, and users can point that to vertex arrays however they’d like. That way it’s not completely useless :persecutioncomplex:

Plus we don’t have to deal with the whole OpenGL version debacle, that I complain too much about.

Float (/double) for vertices/normals/uv and int (/short) for indices?