LWJGL3 and Android Redux

Engines, Libraries and Tools
#1

Hi @spasi. I’d like to continue the discussion from the previous LWJGL3 & Android thread which is locked now.

As more or less expected Google is declining to make a Vulkan Java binding for Android. Even giving what I think is a weak answer:
There are no plans to do a “pure” wrapping of Vulkan in Java. Vulkan would make a poor Java-language graphics API, and the overhead of JNI would mean losing many of the benefits that it’s supposed to provide.

I doth protested, but it’s not likely Google’s position will move. There are no more Java champions on the graphics side willing to take a stand within the Android team. This matches the sentiment I got from a light probing earlier this year. Hopefully they absolutely don’t create some new API on top of Vulkan as mentioned in a follow up to the above quote.

Anyway… I am 100% down to collaborate and help create and maintain a LWJGL3+ port for Android. Don’t worry about the API level needing to be high. It might even have to be API level 24/25; Android N and that’s fine. I do have some complex direct GL/ES code with the video engine and that’s even kind of neat because it would really be pushing things to get LWJGL3 setup to drive it (if possible). We can’t let this go unused! ::slight_smile: I’d be fine switching to LWJGL and doing an Android N release of the video engine if it could actually work; would have lot’s of data then on new device compatibility! Too bad I’ll have to stick to GL/ES, but it’d be interesting. I’d be glad to get some GL/ES and Vulkan demo code out there. I already have an Android GL/ES starter available which I’d rewrite for LWJGL3 via GL/ES in addition to a potential Vulkan version. In this effort I actually provide improved facilitating framework code where the Android API is long in the tooth.

I’ve got tons of devices with each generation of GPU and would be glad to test, refine, and work on any necessary optimizations.

It seems like Android N / API 25 w/ the switch to OpenJDK would be the first to get stable w/ GL/ES and Vulkan paths. I know there is a bit of sun.misc.Unsafe usage and / or some JNI fallback code for JVM optimization. Even if things started with Android N that’s great as there is a huge hole opening up on Android for performance graphics and LWJGL and can fill it.

So since I only have a cursory overview of LWJGL3 from poking through the source code I’d be curious to hear what you think will be the pain points? How are you handling JDK9 removal of sun.misc.Unsafe?

I think the first thing is to evaluate what’s potentially necessary / missing for an easier port. Maybe the Android team would be slightly helpful if there is something missing that could potentially be exposed in the future if no workarounds exist. Just kind of thinking out loud at this point… I’m down to make it happen though.

#2

Hey, quick reply (I had a minor operation today and am in a bit of pain atm):

I still haven’t done any work on ARM/Android, but I have new information to share. @KaiHH has been working hard to make JOML perform nicely on Android (it’s being used in Samsung’s GearVR SDK) and I asked him to do a bit of testing for me. Turns out that, indeed, starting with Android N and the OpenJDK-based SDK, the current LWJGL design might be viable. Unsafe seems to be a lot faster at least.

[quote=“Catharsis,post:1,topic:58057”]
That’s great, will be very helpful.

[quote=“Catharsis,post:1,topic:58057”]
LWJGL 3 depends on Unsafe and escape analysis for extreme performance. Workarounds may be possible, but at the expense of clean code.

JDK 9 does not remove sun.misc.Unsafe and it’s available to all modules (there’s a new, much more powerful, Unsafe in 9 that is available only inside the JDK). As of this commit LWJGL 3 also works on JDK 9 b148+, which introduced a lot of significant changes to the module system.

#3

Get well!

I do look forward in how this can proceed and to start can probably squeeze out ~10 hours a week to help test, but certainly would be glad to get up to speed on the LWJGL architecture / build process in time. If there was a time to get this going though it sure does seem like now! ;D

#4

I will support you as much as I can, but my time and skills are a bit limited in this area. I can however provide pretty big test programs once my Vulkan abstraction makes some more progress.

#5

Thanks to the JDK-9 module workaround by @Spasi, JOML now also runs Unsafe-accelerated under Java 1.9 (as well as the already supported Java 1.4 through 1.8 ) on Windows, Linux and OS X.

#6

Hi @Spasi… I hope you are feeling better!

Just dropping a note that I’ll be attending GDC for the GL / Vulkan sessions and tutorial day. There will be a talk on native usage of Vulkan / Android, but I’d be glad to submit a lightning talk (5 min) and perhaps have the chance to drop the details on LWJGL support on Android & icing on the cake if Vulkan is supported. Regardless if the lightning talk comes through I’ll be able to chat with many high level folks about said support if it was possible to get things working before then. Alas it’s 5 weeks away, so a bit tight on the timeline. I just bit the bullet and got a tutorial GDC pass.

If you could perhaps provide some guidance on where to start or what needs to be done I’ll try to help out however I can. I can probably put in ~20 hours a week starting next week to help for the following 2 weeks then perhaps jam full time on things the remaining time as necessary to get things stable.

I’m finally getting a new dev laptop tomorrow and will be setting up the latest Android Studio setup in the next couple of days. What else will I potentially need? I assume getting comfortable with building LWJGL on the new lappy will be a good start (MacBook Pro in regard to environment). If there is any info regarding this process do point me to it as I didn’t readily find anything on the LWJGL web site.

It’d be absolutely fantastic to get LWJGL 3 on Android and I’ll pimp it hard at GDC and inform folks who is behind it all too (err not me!). ;D I’ll also foot the cost to make a nice color business card / flyer about LWJGL 3 w/ Android support info and pass it out to everyone I meet; will also include JGO link too!

#7

Hey @Catharsis, here’s a plan for what needs to be done:

  1. Decide which ABI(s) we’re going to support. I’d be happy with arm64-v8a and nothing else.
  2. Cross-compile LWJGL natives to the target ABI(s).

This is the part I need more help with. The LWJGL build process uses Travis CI for the Linux binaries. The source code for each project is under the LWJGL-CI account on GitHub. Each repository has two branches (one for Linux x64 and one for macOS) with an appropriate .travis.yml script that performs the corresponding build. We’ll need the same thing for Android (and Raspberry Pi builds later). Basically, a new branch must be created (e.g. master-android-arm64-v8a), which is exactly the same as the other two, except the contents of .travis.yml. The difficulty lies in getting Travis CI to cross-compile for our target ABI.

  1. Change LWJGL so that it supports the new platform/ABI. (should be easy)
  2. Test, fix, test, etc. (someone with Android dev experience: please contribute any information you might have!)
  3. Deploy the new platform/ABI to Maven and the LWJGL site. (should be easy)

If this gets hard, we can focus on some LWJGL modules and skip the rest. At a minimum, we’ll need:

  • LWJGL core
  • dyncall (the core depends on it)
  • GLFW

Good to have:

  • jemalloc
  • OpenAL for audio

When building the core, unnecessary bindings can be disabled in /config/build-bindings.xml.

We should probably move this discussion to GitHub or the LWJGL on Slack.

#8

Err, no OpenGL module in that list?

Also, I assume there’s no hope for Vulkan support on Android?

EDIT: And even if Vulkan is a no-go, I’d still use the sh*t out of LWJGL3 with OGL ES on Android.

#9

OK… I’ll start taking a look ~mid-next week. I joined the slack channel as “typhonrt”. I think starting with the core / bare minimum to get GLES up first is a good idea for sanity then add Vulkan and worry about audio later. I’m generally familiar with the process you outlined and will embrace the black screen of death until I make it blue… :o Hopefully Travis is up for it as with a cursory review it seems to indicate it may be a bit raw.

#10

Just a quick note… most RPis out there will be running a 32-bit Linux OS rather than a 64-bit one that the chips are able to run. Not quite sure what the implications are but I bet that means having to support a 32bit ABI for ARM.

Cas :slight_smile:

#11

[quote=“theagentd,post:8,topic:58057”]
I’m strictly talking about non-JNI code that needs to be built for Android. JNI code, especially in LWJGL, is trivial and can be easily built on any toolchain. Also, APIs like Vulkan have no dedicated JNI code in LWJGL; dlopen, dlsym and you’re good to go.

[quote=“princec,post:10,topic:58057”]
Yes, noticed that in the quick look I had. There seems to be ongoing effort to support 64bit, but who knows when that’ll be ready (RPi 3 only too). In any case, it is my understanding that Android binaries require a special toolchain that is not Linux compatible (correct me if I’m wrong please), so we’ll need a different ABI for RPis anyway.

#12

If you had it working on Pi… would it be “headless”? As in, takes over the framebuffer? (I don’t want to boot into Gnome or XFCE or whatever it’s called)

Cas :slight_smile:

#13

[quote=“princec,post:12,topic:58057”]
Afaict, there are about 5 functions that must be called to launch EGL in “headless” mode. This is a trivial issue, these functions can be added to LWJGL in an afternoon. Or even called via dyncall without bindings.

The main problem is, as I said in the LWJGL forums, handling input without X. Keyboard, mouse, controllers, touch, we’ll need something to handle all that. We cannot use GLFW, it supports 3 backends (X11, Wayland, Mir) and even though Wayland is supposed to be a lightweight solution (and works on RPi afaik), I think you’d want to avoid that too. So we’ll have to go low-level. This is probably not that big of a deal, I mean, the main problem is identifying the APIs we need bindings for. Once we know how it’s done, adding the bindings to LWJGL is going to be quick.

Which brings me to; I don’t own an RPi and I don’t know when/if I’ll have time to do the above research. The easiest thing for me would be to get a list of APIs to add to LWJGL, from someone that knows what’s necessary.

#14

I’ve got several Pis - one of each model in fact - and the time and wherewithal to test if you need me to.

Don’t let input get in the way of development for now - produce a “dud” backend for it that simply fails to connect any devices successfully or something. Then at least I could get on with testing EGL.

Cas :slight_smile:

#15

@spasi would the work I did on android and arm build before help here?

#16

[quote=“princec,post:14,topic:58057”]
Alright, will let you know once we have working ARM builds.

[quote=“ziozio,post:15,topic:58057”]
Yes, your config here should be a good start. Have you tried doing the same for any other library? I guess the hardest would be GLFW and OpenAL-Soft. They have plenty of dependencies other than the basic C libraries, which may or may not be readily available in a cross-compile toolchain.

#17

Hi Spasi, how are the ARM builds coming along? I should have time this month to play with the Pi. There’s a good possibility we might be able to develop a passive display product based on a Pi+screen here.

Cas :slight_smile:

#18

@KaiHH did the first successful build of lwjgl-core + bindings last night. We now have an android branch and I plan to start working on it asap. One dependency has also been built on Travis-CI.

The good news is that it’s looking easier than we thought. The not-so-good news is that we’re talking about Android builds. The Android NDK is supported out of the box on Travis-CI, but afaict we’ll have to configure a cross-compile toolchain manually for generic Linux-ARM builds. Any help on that front would be much appreciated.

#19

The glxgears demo, ported to GLES 3 with lwjgl3 and JOML, running on…

Nexus 6P:

wC3LbPgtVY0

Nvidia Shield:

3v9jpelfX6g

#20

The same demo on a HTC Nexus 9.

JpF_EdyQmWY