Most of those listed there are tech demo’s, using old engine code, abandoned, unfinished or WIP’s.
More than half of them are using JMonkeyEngine 3. None are abandoned as far as I know. A few of them are WIP. Boardtastic, Fleshsnatcher and 3079 aren’t unfinished. The only pure tech demo is ArdorCraft. Spaced is unfinished but already very complete, it’s much more than a tech demo.
The word here is: underwhelming.
That’s generally the issue with 3D games by indies… quality content production is the bottleneck, not necessarily the engine.
The only real shortcut is picking a specific visual style and focus on gameplay, or go 2D like a sane person.
I’d say there are many dimensions to this and I’ll answer from the perspective of what I’d build…
For outdoor environments I’d focus on octree partitioning with an emphasis on texture / geometry streaming and further tessellation for enhanced detail. For indoors / non-destructive you simply can’t do better than portal/bsp/pvs to my knowledge. One could still support some level of streaming & dynamically loading BSP and other data for expansive indoors environments.
Technologically speaking especially for outdoor and streaming data parallelism is crucial. This means supporting multi-threaded GL and CPU side architecture. From a Java perspective I personally would really like to start using the Disruptor on the CPU side of things which would handle anything routing between the disk and the network to threads streaming through to GL. I am bit sad though that the Disruptor doesn’t work on Android due to its reliance on sun.misc.Unsafe. If Android is a target and it should be this means sticking to OpenGL ES for the time being though that could be different a couple years from now and things may properly converge to just OpenGL across all mobile SoC/GPU vendors. This means though supporting OpenGL ES 3.0 and the GL threading API and while one is at it all the other goodness 3.0 brings.
OpenGL AZDO is important of course. There are many aspects there that support texture streaming / binding concerns. I’d figure out the limitations of MultiDrawIndirect. I really wished that the first Android Extension Pack covered AZDO extensions. I’ll be able get started with them on the K1 though since it supports OpenGL 4.x too. Though I’ll have to think about how the majority of the engine work must remain compatible with GLES 3.0 right now.
Compute shaders and OpenCL is very important. Particle engines will all be compute based. In fact the first demo I’m going to port to Java is this one:
http://docs.nvidia.com/gameworks/index.html#gameworkslibrary/graphicssamples/opengl_samples/computeparticlessample.htm
For other modern GL demos check out all of them:
To have any chance at maintaining an engine now and into the future modularity must be a concern from the get go or go the way of all past Java engines. Component architectures (CA) are key in this direction IMHO. If you don’t think so, well, I can’t help you there much. 
A CA is flexible to support structuring both the engine subsystems at a high level that is compatible with data parallelism in addition to other aspects like entity systems. It also happens that it supports various rendering paths and hardware diversification concerns which is quite important for long term evolution of an engine. It is what is going to make relatively painless my efforts to support OpenGL ES 3.0 for the majority of Android devices and have a GL 4.x path compatible path with the K1. It also makes tooling possible hence the nice user experience Cas also mentions when he speaks favorably of Unity.
I’m so excited that I get to start down parts of the rabbit hole above tomorrow when my Shield Tabby arrives… 
Gameplay is king at the end of the day. From an indie perspective one could consider procedural content generation and have great gameplay.
The first example that pops to mind in regard to picking a specific visual style and procedural content is Love:
http://www.quelsolaar.com/love/
I never have gotten around to checking it out though so, I’m not sure about game play.
What I think would really benefit the larger Java game dev community is as many folks as possible getting their hands some of the leading tech today (Right at this moment on Android that is the Tegra Shield Tabby, desktop there are various high end GPUs to pick) and start building new engines that may not be compatible with the wider ecosystem. OpenGL ES 3.0 should be the absolute minimum supported for new engine development right now and experiments with OpenGL 4.x should occur on mobile and if you are on the desktop only ignore anything that isn’t the latest GL 4.x point releases.
…and jPCT/jPCT-AE are being ignored…as usual. Anyway, 3D for desktop Java isn’t very common these days, but for Android it is. For jPCT, i would estimate that the ratio between desktop Java and Android projects is somewhere between 1 to 10 and 1 to 20.
I really liked this blog post:
[quote]Today I was chatting on twitter about an engine and some smart guys posted a few links, that I want to record here for posterity. Or really to have a page I can point to every time someone uses a scene graph.
[/quote]
Some scenegraphs support cells and portals, JMonkeyEngine supports BIH trees:
https://jmonkeyengine.googlecode.com/files/Scene%20Partitioning.pdf
jKilavuz can be used with JPCT to compute portals and “sectors”:
http://www.jkilavuz.com/features.html
IsoSurface demo uses JMonkeyEngine 3:
I don’t find it ugly.
Sure. That’s not the context of my comment which was targeting “tech”.
A game industry TD who links to other long time game industry TDs and leads…who’s gonna listen to folks that that? They must all be old and set in their ways or something.
You criticize JMonkeyEngine whereas “opengl.gui” within Clyde GUI is derived from … JMonkeyEngine Banana User Interface (JME-BUI):
Anyway, where I was going with this…
were one to start from scratch with an idea to make an engine that might compete, say, with Unity - at least on the desktop - what might be an overall design for it? Given that all these different techniques exist - such as BSPs, octrees, portals, scenegraphs, etc - what overall architecture might be able to throw everything in and get it working together?
Cas 
To get anything close to Unity functionality from the tooling perspective and runtime flexibility a component architecture is pertinent.
Unity itself licenses and integrates various 3rd party middleware. SpeedTree being an example. If your effort was commercial I’d look at various 3rd party industry standard middleware to integrate. Sundog’s SilverLining (sky rendering) and Triton (ocean / wave rendering) could be good additions to SpeedTree.
For indoors I’d do a patchwork of separate BSPs (to my understanding this is what the Unreal engine does; haven’t looked at Unreal 4 / recent source) which can be streamed.
Outdoors indeed would be Octree plus heavy streaming… Try and work out an editor like Grome (http://www.quadsoftware.com/index.php?m=section&sec=product&subsec=editor). Don’t know if that company would license editor code.
Pulling together an editor for outdoor / indoor would be a challenge.
I prefer not talking about component architecture as this is something I don’t know enough to judge whether it’s relevant or not for gaming.
I challenge you to find an intelligible open source implementation of portal culling and “portalization” (creation of the cells and the portals) whereas there are tons of examples of partitioning trees on the web. Grome is used in some commercial games to create realistic terrains, it seems to be quite good.
I can speak about what I really know. Whatever you will do, you’ll need to use partitioning trees, even to generate cells and portals. If your geometries don’t change a lot, partitioning trees should be enough, they are generally faster to visit but modifying them can be sometimes very expensive. If you want to manage fully destructible environments, you can go on using trees and cheat a lot (manage destructive objects separately and keep a static tree) or you can use a graph of cells which is more expensive to visit but a lot less expensive to modify.
Some other kinds of culling might be interesting, I think about back face culling, contribution culling, view frustum culling and occlusion culling (with anti-portals). Back face culling and view frustum culling are often supported by major 3D engines written in Java.
When you are talking about backface culling do you mean precalculcated lists depending on view direction or just gpu flag?
These are all solutions to unspecified problems. What are the problems? What queries do we need to be able to perform on the data?
For example: we want to draw the diffuse lit scene. We need to be able to efficiently know what is potentially in the scene; so we have a frustum cull.
Another example: we want to know if two “things” collide. What are things? How many of them are there? Do we need perfect collision or approximate collision?
And so on.
Cas
I was talking about that which is very rudimentary. Facet culling is disabled by default. Personally, I prefer using separate meshes and the OpenGL flag, for example when entering a building.
@princec Something outside of the view frustum might influence the scene, for example a source of light, a lamp.
There are several kinds of tests that can be done to avoid doing useless heavy computations when looking for collisions, you can benefit of spatial subdivisions and bounding volumes to eliminate obvious cases. Imagine that the player is the entrance of an hotel at the ground floor, you don’t need to check whether he collides with an enemy in the 6th floor.
You can detect collisions a priori or a posteriori, you can use a physics engine (JBullet, Jinngine, …) if you don’t want to reinvent the wheel. Keep in mind that if two things make a move greater than their respective bounding volumes between 2 instants, you might miss some collisions when using a posteriori collision detection and multisampling can become very expensive if they move very quickly. Sometimes, we can cheat, you can use a simple ray to represent the trajectory of a bullet but it becomes really weird when shooting a guy very far from the player.
The sweep-and-prune algorithm allows to avoid testing collisions between all possible combinations of objects:
SAP is basic, easy to implement, brute-ish force and can be more than reasonable.
Well, physics and such notwithstanding… let’s assume for the moment we’d just want to deal with rendering a scene.
Cas
Unity is quite a beast.
The overall architecture seems really well thought, at least from the outside. To me, its biggest advantage is that everything in a scene is component-based, it gives an amazing amount of flexibility, both for us users, but also for the Unity dev team. They just keep adding new components version after version, such as the somewhat recent 2d physics and 2d rendering components, and soon the reflection probes.
In terms of rendering, I’m not sure if that answers the question, but they use a mix of frustrum culling, static batching, dynamic batching, and occlusion culling (using Umbra). For the underlying problems all those solutions are trying to solve, I’d start looking at the theory behind each of them.