There’s a new lmdb branch in the LWJGL 3 repository that implements bindings to LMDB. It’s an embedded key-value store that may be used for all kinds of persistence, from game settings to shaders, models and textures (binary values up to Integer.MAX_VALUE in size are supported).
See this presentation to understand what’s special about LMDB.
See this LWJGL sample for a demo of the API and an example of zero-copy string persistence.
The nightly builds do not include LMDB yet, you’ll have to clone LWJGL and build locally to give it a try. I’ll decide what to do after the 3.0.0 release, but it’s definitely small enough to embed in the standard LWJGL distribution. Please let me know if you’re interested.
I understand what the LMDB is, I can think of non gaming scenario’s where this is very useful. But what do you see as the use case in a gaming sense for a database (and one that is super fast)?
[quote=“ziozio,post:6,topic:56942”]
I mentioned some ideas in the first post. I often see game developers creating custom solutions for storing user settings, game saves, metadata for shaders/models/textures, game levels, etc. I think replacing all that with a robust, easy-to-use and very efficient database like LMDB would be useful. It can even be used to store binary blobs (meshes, textures, sounds, shader binaries).
It seems to me that a lot of this shouldn’t be included into core LWJGL. I am a minimalist and would love each sub library of LWJGL to be a separate .JAR or a plugin. I have checked out some of the posts from you Spasi, but I can’t really follow along with what they say because they lack “this is what this is” but just “here is why we are doing this and whats to come of it.” So I am not in the now, like you seen in another post.
I might never migrate to a newer LWJGL than before you added a bunch of stuff. I might actually rip a lot out of the jar if thats the case.
I’m slightly shocked to discover I agree with something @Hydroque says! I’m really not at all sure how this fits in with LWJGL, either in a gaming context, a general purpose graphics context, or even in a Java context… I’m very reluctant to bind to native code to do stuff that Java code can do. The LWJGL was all about binding to native APIs that could not otherwise be done in Java - minimising the area of the native interface footprint to the bare minimum to get the job done - and it seems there’s a bit of “kitchen sink” syndrome going on.
I think that’s the reason why Issue #100 exists, for users to be able to exclude what they don’t need.
There was recently also an idea to host a little web application on lwjgl.org where users could select which modules/dependencies they want, which then get bundled into a custom zip download or even an IDE/Maven/Gradle/… project folder.
At least I envision it much like the Spring Initializr site for the Spring framework.
This is why I waited for the 3.0.0 release before merging the LMDB branch. I expected some controversy and wanted to see if enough users would find it useful first. Currently LMDB only exists in a nightly build of 3.0.1. No harm has been done. This is also the 3rd time I have to mention in this forum that 3.0.1 will not be released without a solution for #100, which will break the LWJGL distribution in separate artifacts for each binding; you will be able to choose what you want or don’t want in your application.
[quote=“princec,post:10,topic:56942”]
Offering solutions for gaming and graphics has always been LWJGL’s primary objective and what most people in this forum care about. But it’s not only about that. Which makes sense since I’m (almost) the sole developer, I use it a lot in my projects and I don’t make games.
I bet no more than 5 users have ever used LWJGL’s OpenCL bindings, yet I find them super-important and have spent countless hours working on them. Why aren’t there any complains about those?
[quote=“princec,post:10,topic:56942”]
I don’t agree with this either. Just because something can be implemented in Java does not mean it should. I’ve actually done it: JGLFW. It was a GLFW implementation written entirely in Java, using only bindings to OS functionality. It was an experiment to validate LWJGL 3’s approach to JNI and I gladly deleted it when I became satisfied it worked nicely. Why?
Look at the mess LWJGL 2 was in. There were dozens of bugs and no one to fix them. Who’s going to write a decent windowing system in Java (or anything that touches lots of OS-specific APIs like GLFW and LMDB) and maintain it?
A Java binding to a native library means that Java developers benefit from the existing community in many ways: more example code out there to get you started, more users testing the same code, more users contributing fixes. The existing community benefits too, Java developers can contribute feedback and fixes to the native library (there are many examples of that in GLFW already). Overall, the bigger the ecosystem around a library the better for everyone involved.
[quote=“Hydroque,post:8,topic:56942”]
This is possible and some users do it. You can remove any binding package you don’t need from lwjgl.jar and you can also delete the jemalloc/GLFW/OpenAL shared libraries. #100 will make it so that you don’t have to do it manually and will also split the lwjgl shared library itself (which contains the core functionality + statically linked bindings).
I think you’re being a little defensive about my observations, Spasi! I’m quite aware of the 3.0.1 goals already. What I am questioning is that LWJGL is, as a whole, becoming quite a sprawling mess of functionality, and that (as you allude to below), it appears to be largely driven by you and your own personal needs and as such seems to me to have become a giant git repository with all of your open source work in it. This is an important observation, which I’ll get to…
What it definitely isn’t, modular or not, is a lightweight Java games/graphics library! It’s no longer a lightweight project as it covers everything from GUIs to compute to audio to in-memory databases to hacking the Java language to get structs in. It’s not Java any more as it now contains multiple binary libraries. It’s not a single library now but a whole multitude of libraries. So there’s that to think upon.
[quote]I bet no more than 5 users have ever used LWJGL’s OpenCL bindings, yet I find them super-important and have spent countless hours working on them. Why aren’t there any complains about those?
[/quote]
Because it still satisfies the original premise of LWJGL, which is the minimum and most direct binding to a native API possible to achieve something vaguely game-related. Personally I wouldn’t have included it either, but I’ll get to why in a mo…
There are many ways to implement functions, but the question is not which language something should be implemented in, but whether it should be implemented at all. The crux of the matter hinges on a comment you made about LWJGL 2:
[quote]- Look at the mess LWJGL 2 was in. There were dozens of bugs and no one to fix them. Who’s going to write a decent windowing system in Java (or anything that touches lots of OS-specific APIs like GLFW and LMDB) and maintain it?
[/quote]
What LWJGL 3 is now is a vast collection of bindings, most of which are irrelevant to most people. What this means is that maintenance effort is going to be simply non-existent on it, and once you (eventually) grow tired of it, and you will, it’ll just stagnate and get bitrot and become a large legacy which will distract people from otherwise creating separate efforts to solve specific issues.
I lost interest in maintaining LWJGL after a few years of developing on it, largely because it did everything I needed it to by that point, and also because I was seeing virtually no return on my efforts. I watched Markus accumulate two and a half billion dollars on the back of the literally months and months of effort I’d put in on the library he used to catapult himself into the realms of the unspeakably rich with barely so much as a thankyou, and that, for me, was the point at which I decided I couldn’t be arsed with open source any more. I only have so much time left in the world and a lot of things I’d like to do, and making Markus richer wasn’t on the list, so I diverted my efforts elsewhere. At some point, you too will find something more interesting and distracting, and you’ll be leaving behind a huge, huge pile of stuff which no-one except you really cares too much about.
I strongly advise you to split LWJGL up into the bare minimum and then create separate, independent projects that build on top of or besides LWJGL for GUI libraries, structs, OpenCL, in-memory databases, vector math, etc.
It’s a pity to de-motivate the project lead like that. The main currency in open-source projects is respect, fame and the warm feeling of being altruistic and you’ve painted a sorry picture of them all!
Whether or not LWJGL is truly lightweight, it’s open source and can easily be forked.
If it’s also modular as promised, even better. Making separate Git projects out of it is another good solution and can always be done in the future.
I don’t think that Spasi’s additions do any harm and he brings up interesting libraries that I’d never heard of before since they’re not in the java ecosystem, but appear to be best practice in the C world. Putting together a bare bones ‘best practice’ game dev library is useful. I imagine that it’s also easier to manage build scripts and forums and what-not by combining all these projects into one, saving Spasi’s and others’ precious time.
The huge amount of work that Spasi has put into LWJGL3 and the sister-projects should be celebrated. It is judging by his medal count! It’s a shame to criticise him for going even further beyond the call of duty by including extra interesting stuff.
That’s a shame that you feel that way. I think you know there’s more to life than cash and fame, you probably just had a rotten day.
Think of all the mathematicians, scientists, engineers and writers who did amazing things and didn’t get paid a cent for their life’s work. They’re not losers for being hard working and altruistic, they toiled for a purpose above themselves which added to their sense of life fulfillment and contributed to better living standards for everyone.
I mean, why do you think that the super wealthy always create universities named after themselves such as Stanford, Yale and Duke? It’s a cliche, but making a ton of money above the average income doesn’t fulfill life’s higher aims.
As a LWJGL dev you should be proud to have contributed to the sensation that was Minecraft. Markus wasn’t obliged to reflect any cash or fame on the downstream tech that he used. From memory he did have a credits page on Mincraft.net which linked to LWJGL and generated much interest there and also here at JGO, promoting Spasi and princec and all the other LWJGL devs which will help them get work in the future on anything graphics and game-related, in addition to respect in the game-dev community.
I’m sure Markus will drop in one day to remember the good old times and thank everyone who made tech like LWJGL.
Thanks to all the devs who made and continue to make lwjgl, java, javagaming.org, libgdx, and all the other incredible projects 8)
Bindings that I added without anyone asking, but everyone agreed were extremely useful:
jemalloc
stb (image IO, font rendering, vorbis decoding, etc)
Bindings that I added without anyone asking, are not that exciting, but try to solve real problems LWJGL users face:
NativeFileDialog
Nuklear
libstem_gamepad (in a branch, failed experiment, to satisfy concerns about GLFW’s gamepad support)
Bindings that I added without anyone asking, either from curiosity or because I needed them for personal work:
xxHash (tiny & fast non-cryptographic hash functions)
LMDB (post 3.0.0)
ODBC (mostly complete branch)
hwloc (incomplete branch)
I hope it’s obvious that the majority of the work I did was driven by user needs. The only personal binding that made it to 3.0.0 is negligible compared to the rest.
I would argue that it is indeed lightweight because it doesn’t do anything. None of the APIs or functionality provided by LWJGL has been designed or written by LWJGL developers. The only thing the LWJGL core provides is a half-decent way to interact with native APIs and data from Java/JVM.
Also, why is any of the above an issue if #100 allows you to download LWJGL core, GLFW, OpenGL and OpenAL and nothing else?
[quote=“princec,post:15,topic:56942”]
I don’t get this. Why does it matter how many binaries there are? Deployment issues are a thing of the past with the SharedLibraryLoader. LWJGL 2 also had two native binaries (lwjgl + OpenAL Soft), now there are 2 more (GLFW and the optional jemalloc). How’s that less Java than before?
There is a very critical difference. Once Elias stopped working on LWJGL we were screwed, because there was no one with the experience to improve/fix the OS-specific functionality. There’s no expertise required to maintain LWJGL 3. Someone has a problem with GLFW? Open an issue on their github and (if it’s legit) elmindreda will fix it eventually. It’s not our problem anymore.
[quote=“princec,post:15,topic:56942”]
Oh, I do have something a lot more interesting to work on. That’s why I’ve put a huge effort into making LWJGL 3 easily maintanable.
On seeing a return on open-source efforts… I don’t know, that’s not what open-source is about I guess. I even removed the option to donate to LWJGL when I took over. Didn’t feel right without having something to deliver. On the other hand, what we have been doing in LWJGL isn’t just open-source… Writing JNI bindings is on its own level of masochism. It’s the kind of open-source that no one wants to be doing. That usually has value. When my wife asks what I’m doing, I wish I had something better to say than “omg I’m writing Vulkan bindings, do you know how IMPORTANT this is?”. Especially with the economy in Greece being what it is.
On Minecraft, I cannot claim any credit for what LWJGL was when Minecraft happened. Maybe Cas is right, I don’t know. What I do know is that post-Minecraft success, its users had lots of trouble because of LWJGL and we didn’t do much for them. We never had much of a contact with Marcus or their devs either, which I found strange given how critical LWJGL was for the game. Strange situation.
[quote=“princec,post:15,topic:56942”]
Having everything in the same repo, in branches, or in different repos, is just a technicality. The only thing that matters is how the software is deployed. With #100 + a few other ideas we had recently, I believe everyone will be satisfied.
[quote=“princec,post:15,topic:56942”]
I guessed so, I’m just trying to address your concerns in a way that will be understandable to everyone reading. I fully appreciate your input btw!
[quote=“CommanderKeith,post:18,topic:56942”]
Don’t worry, there’s no demotivation going on. Cas has strong opinions and gives honest advice. It’s always helpful.
That’s why I’ve put a huge effort into making LWJGL 3 easily maintanable.