[Tool] DOPE

WIP games, tools & toy projects
#1

DOPE (won’t be the official name :P) stands for Data Oriented Programming Environment and it’s a tool that lets programmers and non-programmers visually build software components. It’s not even close to an alpha version and it isn’t game development specific, so let me know if this post is inappropriate for this forum.


http://www.java-gaming.org/user-generated-content/members/4730/dope.jpg

Technical details:

  • The user interface was built with JavaFX.
  • Currently it only supports T-SQL generation. We had a need for this in my day job and I hacked it in a couple of days. I haven’t started working on proper code generation yet, though most of the design is in place.
  • It’s been designed to support Java, OpenCL and GLSL code generation. It will also allow seamless transitions between execution contexts, without manual coding. A data flow could start in Java, go to CL, then back to Java, then to a database.
  • In a game engine, it could be integrated and used like UE’s Kismet.
#2

This looks interesting. Are you planning on releasing it for testing or something like that?

#3

Yes, when it’s able to generate and execute JVM code (at least). At the same time I’ll put it on github and open source it, most likely.

#4

This looks really slick! Can’t wait to try it out :slight_smile:

#5

I’ve always questioned just how effective software as “circuit design” is for real non-programmers. (MIT had/has what appeared to be something more interesting from a 30-sec peek…can’t remember the name). Stuff like this is very nice however for content composition/creation tools.

#6

I hope it can work with additional library too.

#7

I’m looking at it from another angle: would laying out concepts like this show me non-obvious ways of arranging things for extremely high performance?

Cas :slight_smile:

#8

[quote=“Roquen,post:5,topic:39752”]
It’s pretty effective in the DCC world. The best example I know of is Softimage ICE, which is actually what inspired* me to create this tool. It’s very popular and I’ve seen non-programmers do crazy things with it, real programming stuff. Also see Houdini’s procedural workflow, the shader trees in MAX/Maya, Kismet in UE, etc. Maya is getting a tool similar to ICE in a future version too afaik.

Also keep in mind that the tool itself is intended to be used by developers only. Whether or not non-programmers have access to it depends on the product and how the tool has been integrated. One obvious concern is safety; you don’t want non-programmers to have access to file access nodes etc. So, a developer would have to filter and customize the core tool and expose something more usable by non-programmers (or programmers with restricted access). Hopefully this process will be easy to perform by the tool itself.

wrt: MIT’s Scratch and similar tools. Nothing comes close to general purpose programming and DOPE is meant to enable that.

  • Even though the underlying design is completely different, I shamelessly copied the UI to avoid having to come up with “programmer graphics”. I was also learning JavaFX at the same time and wanted to push it hard and see what it can do. Anyway, the current GUI is a placeholder, it will change while I’m finishing the core functionality.

[quote=“ReBirth,post:6,topic:39752”]
The plan is to expose everything in an API, so you’ll be able to use it without the GUI. I also have a 3rd-party willing to create an HTML5 client (think client-side design -> server-side code generation + execution).

[quote=“princec,post:7,topic:39752”]
The main performance benefit of using a code generating tool like this is code reuse across different execution contexts:

  • You design an algorithm once and it can run on both a JVM and a GPU. Not only you didn’t have to write it twice, but you didn’t even have to bother with learning OpenCL/CUDA/whatever programming. It will just work (assuming of course you stay within certain limits - think Aparapi’s constraints).
  • There’s a benefit from the user perspective as well: A user with decent GPU and working drivers will get the performance benefit transparently. A user without one, will still have a working app, the runtime will simply fall-back to multi-threaded JVM execution.

Also, runtime code generation means the tool can make optimization decisions based on the real user environment. Again, the developer doesn’t have to intervene or even think about it, advanced stuff like tuning for a GPU’s warp/wavefront size happen automatically.

But the answer to your question is yes. The main reason is that a graphical programming environment opens up a lot of possibilities for visual debugging:

  • Since the tool will have full information on the input data layout, it will be able to display intermediate values in a data flow for multiple input values at once. Even without real data, you’ll be able to create additional nodes that generate dummy data and feed those in to see how your algorithm behaves. Softimage ICE does this very nicely with multiple options for how the data is visualized in the 3d scene.

  • Break-points and step-by-step debugging can be so much more powerful. See how amazing it looks in Kismet.

  • Using graphics and code-generation for programming makes it much easier to implement stuff like Bret Victor’s brilliant ideas.

#9

@Spasi: Yeah, like I said content creation is one example of a good fit. My “musing” was more about joe average non-programmers instead of artist, scientist, engineer non-programmers.

@Cas: I pretty much draw stuff like this on paper for every difficult SIMD problem I have.

#10

I really liked Kismet back then when I was playing around with the UDK. Nice work!

#11

I’m really intrigued to have a go with this. I’m a great fan of higher-level visual programming / patching environments (hence Praxis), but somewhat sceptical of graphical programming for low-level, general purpose stuff. Maybe I like editing code too much! :slight_smile: Still, something that offered seamless transition back and forth from graphical view to code view would be interesting - perhaps the mutant offspring of DOPE, Light Table and Field?

The interface looks great btw (really must do some work on Praxis’ graph layout - it’s looking a little tired in comparison! :wink: )

#12

Nice!
I need to use a node system like this one in a current project of mine.

Is it plain JavaFX, or did you use a library for handling the visual aspect of the node system ?
Do you have experience with doing the same, but in java 6 ?

#13

[quote=“nsigma,post:11,topic:39752”]
I have considered this and may do it later on. But the code view won’t be the final generated low-level code. It’ll have to be some DSL built on top of the core API, so I’ll have to first finalize that to see what can be done. In any case, it will be easy to call out to custom Java code, so if some functionality is missing or is suboptimal, workarounds will be possible. I also plan to implement much of the tool’s functionality with the tool itself and expose it to the user. This means that when you aren’t happy with something you can simply open a node tree and do a fix that modifies the tool’s behavior.

[quote=“arnaud_couturier,post:12,topic:39752”]
Yes, it’s 100% JavaFX and everything was built from scratch.

Everything you see can be implemented in Swing/Java2D of course. But it’ll have to be a lot of effort to reproduce the node canvas. JavaFX is a scenegraph, which provides a lot of functionality out of the box. You can think of your UI as a 3D scene with nested coordinate systems, to which you can apply any kind of transform, etc. This simplifies node layout and makes stuff like zooming in/out, dragging a connection from one node to the next, etc. really easy. I should probably capture a video of this, the canvas is really dynamic.

Btw, this was all done during the first 3 months of 2012. Back then the SceneBuilder hadn’t been released and JavaFX didn’t have the canvas node (allows drawing stuff like Java2D). SceneBuilder (+ FXML) is great for generic UI layouts, I’ve used it in another project and I’m very happy with it. Canvas would have been great for the minimap (on the top right corner), right now it’s basically another scenegraph of rectangles, which is kinda overkill.

#14

Here’s a demonstration video.

#15

Just… Awesome…
When will this be out? :OO

#16

Sheesh! You’ve virtually made your own CASE tool…

#17

Alive? Dead? Somewhere in the middle? (Just curious).

#18

I’m taking a break from developing it, for a few reasons:

a) When I’m not busy with my day job, I’m working on LWJGL 3. This is much less exciting for me, but it needs to be done asap.

b) I’m waiting for Kotlin to mature. The current codebase is already huge and after evaluating Kotlin I’m certain I can rewrite it down to 1/4th the size and double my productivity. This won’t affect the API users (it will still be Java), but I’m also confident that a Kotlin DSL will be a MUCH better way to interface with the library.

c) I’ve had a few options for funding this and building a small team to help me finish it. I know I can’t do everything on my own in a reasonable amount of time. But I haven’t decided yet and there’s also the kickstarting craze I could look into.

Anyway, it’s alive and I’m heavily invested in this project. I’ve given up on trying to make games long ago. :wink: