Scar: Simple Java-based build system

Shared Code
#1

I’ve been meaning to make my Java build system stuff available for other to use for a long time. This plus a recent forum discussion plus my trying to re-sign/pack/LZMA LWJGL applet stuff led me to finally clean it up. I have tentatively named it “Scar”, which is not only bad ass sounding, but reminds me of what ant/maven/etc has done to my soul. I’ve documented the project, but not yet written complete javadocs. I don’t yet have hosting (waiting on Google Code to let me use the name), but here is the documentation and a download link:

http://n4te.com/temp/scar.html

I’d love to hear what you guys think! Don’t hold back. :slight_smile:

There isn’t a whole lot to it, but if you are going to write your build system in Java anyway, Scar should minimize the amount of work you need to do. Writing the build in Java has a lot of advantages, such as simplicity, developer familiarity, and the ability to fully debug the build in an IDE. If writing your build in Java doesn’t appeal to you, it should be easy to leverage Scar using a different JVM language. Scar pretty much replaces ant completely. It doesn’t have artifact repositories or plenty of other stuff I’m sure is out there, but it seems like maybe that stuff could be tacked on, resulting in a simpler overall system.

Here’s one of the utility methods, showing off how much functionality is available without a whole lot of typing:


static public void signLwjglApplet (String dir, String keystoreFile, String alias, String password) throws IOException {
	for (String jarFile : new Paths(dir, "*.jar")) {
		sign(normalize(unsign(jarFile)), keystoreFile, alias, password);
		if (fileExists(jarFile + ".lzma")) lzma(jarFile, jarFile + ".lzma");
	}
	for (String jarLzmaFile : new Paths(dir, "*.jar.lzma")) {
		if (fileExists(substring(jarLzmaFile, 0, -5))) continue;
		String jarFile = sign(normalize(unsign(unlzma(jarLzmaFile))), keystoreFile, alias, password);
		lzma(jarFile, jarFile + ".lzma");
		lzma(pack200(jarFile));
	}
}

I need native launchers, so you can add the Scar directory to your path and run it by typing “scar”. Does anyone have a basic template for this? I should use BAT for Windows and SH for everything else?

#2

http://www.java-gaming.org/index.php/topic,22824.msg188572.html#msg188572

#3

Hmm, 1 line SH file? Nah, can’t be right. Ant’s SH file is 326 lines. :stuck_out_tongue: Ok, so the file is trivial, what about permissions? Is it possible to zip the SH file on Windows and have it be executable when unzipped on Linux?

#4

Looks really cool, thanks for sharing 8)

One thing that made it look attractive is not having to stuff around with XML files like in Ant. But then on reading the documentation you’re using YAML. Can the API be used without any config files, just in pure code?

PS: This lib would make LWJGL so much easier!

#5

Thanks! :slight_smile:

Yes, it can be used solely through code. See the 2nd paragraph under the heading “Projects”. However, I think you will find the YAML more convenient.

The idea with the YAML is that it makes it easy to include a “project.yaml” file to allow Scar to build your project without any other effort. Ant absolutely sodomizes XML. Scar’s use of YAML is much different – it is solely a project descriptor. It only provides information about the project, there is no goddamn scripting or other crazy stuff. :slight_smile:

The YAML and Project class are similar to Maven’s “project object model” (POM), except there is no schema – the Project class is totally generic. Only if you chose to use Scar’s utility methods are there any requirements about what project properties need to be defined. If you wanted to go off and write your own build system code, you could define your own project property requirements. Doing this isn’t really all that crazy either, since the Scar utility methods only average maybe 20 lines of code each. The Project and Paths classes go a long way toward simplifying the build code.

[quote]PS: This lib would make makes the whole LWJGL thing so much easier…
[/quote]
Fixed. ;D I hope Scar will make both JWS and applets easier, so there will be less of an excuse for people to only post a JAR to show off their stuff. It can already completely package a Java app for JWS, including generating a JNLP, using pack200/gzip, and serving the files with Apache. The entire process would look like this (this is signing, packing, everything, without dicking with keytool, etc):


Project project = project(".");
clean(project);
compile(project);
jar(project);
dist(project);
jws(project, true, "Esoteric Software", "Some Game");
jnlp(project, "http://esotericsoftware.com/somegame.jnlp", "Esoteric Software", "Some Game", "splash.png");
jwsHtaccess(project);

Maybe the coolest part about Scar is that it uses 3 of my OSS projects. 8) The game I’m working on uses 6 of my OSS projects. I think maybe I should stop building tools and spend more time on the damned game… :persecutioncomplex:

#6

That’s great then. Thanks for making it clear.

The one-jar option is a good idea. It’s such a pity that WinZip and friends steal the .jar extension so double-clicking a jar file usually opens WinZip and not the jre on many computers

#7

looks great and very useful, will hopefully give it a try soon.

#8

[quote]The one-jar option is a good idea. It’s such a pity that WinZip and friends steal the .jar extension so double-clicking a jar file usually opens WinZip and not the jre on many computers
[/quote]
Very annoying. Its like stealing the .docx and opening as a zip… So you can’t rely on jars.

As for this library it sounds like something I need. I always found that getting a finished product out was a lot of hassle. So I’ll surely be looking at this later.

Maybe some IDE’s should use your library…

#9

Scar now has an official home! Yay!
http://code.google.com/p/scar/
I’m glad Google was so nice to let me have the “scar” project name. Until now, I have never been able to get the cool names (I end up with names like KryoNet :-*).

Documentation has been improved. I’ll have the code and a download up soon.

#10

Well, this is really getting interesting. ;D I looked at Janino and it sounds nice. I didn’t dig in its source, but I assume it is put together better than BeanShell. I was bummed Janino doesn’t have support for some 1.5 features, namely enhanced for loops and varargs. Apparently BeanShell does, but then again BeanShell sucks. :stuck_out_tongue:

So, what to do? I found that with ~35 somewhat cryptic lines, I can just compile and execute a string in-memory using only the JDK. :o Sweet!

YAML has nice syntax for multiple documents in one file (—). Now you can simply supply a second YAML document to customize the build. It looks like this:


source: src|**.java
main: com.example.MainClass
---
System.out.println("How cool is this!?");
clean(project);
compile(project);
jar(project);
dist(project);
jws(project);

The way it works is it wraps your code with this:


import com.esotericsoftware.scar.Scar;
import static com.esotericsoftware.scar.Scar.*;
import com.esotericsoftware.minlog.Log;
import static com.esotericsoftware.minlog.Log.*;
public class Container {
   public void execute () {
      // Code from YAML goes here.
   }
}

Edit: Actually, I made it even cooler than that… the method is Scar.executeCode(String code, Map<String, Object> parameters). The parameters are used to output the appropriate imports and the “execute” method declaration, then the values are passed to the execute method so they are accessible for the code string being executed.

#11

Javadocs are done, documentation is done, download is up, simple logo is up. Project is complete for now!

#12

Well, I did a little bit more work. I added a “classpath” statement, so that build code can pull in JARs for the build process. Fixed a bunch of bugs too.

I am now building my game (codename: Dragon) using Scar! The result runs from JAR and also JWS. Build file looks like this:

dependencies:
- ../skorpios-desktop
- ../kryonet
- ../yamlbeans
main: com.esotericsoftware.dragon.DesktopDragon
---
buildDependencies(project, args);
clean(project);
compile(project);
jar(project);
dist(project);
jws(project, true, "Esoteric Software", "Dragon");
jwsHtaccess(project);
jnlp(project, "http://esotericsoftware.com/dragon/dragon.jnlp", "Esoteric Software", "Dragon", null);

There are other dependencies that get pulled in because kryonet depends on kryo which depends on minlog, skorpios-desktop depends on skorpios-common, etc. When I run Scar on Dragon, it automatically builds all the other projects first and sucks in the dependencies. 8)

It is still too early for my official alpha, but here is a little sneak preview to prove I’m not full of shit with the whole Scar business… :wink:
http://www.esotericsoftware.com/dragon/dragon.jnlp
When you run this it will start a server on your machine and a client. Go ahead and run it again and it will start another client (but not two servers). You get a random character and can move around and hit others. If you die your character is gone and you might as well close that client. For my official alpha unveiling I’ll run a centralized server so we can beat on each other.

Actually using Scar to build my projects has helped flesh it out quite a bit. It could really benefit from a few others doing the same. I think it is already very close to being a generic solution for most projects, but could maybe be improved based on how people set up their projects and/or how they want to customize.

I’ll add an applet build next, then I’ll go back to working on my game!

#13

Some thoughts:

Apparently BeanShell does, but then again BeanShell sucks. :stuck_out_tongue:

whats the problem with beanshell? Seems like a natural fit for this project?

buildDependencies(project, args);

  • Having to pass an unknown argument like “args” is a bit “unobvious” :wink:
  • Make project a global, imho the yaml file is the project, so why have to pass it to every target

On a personal note:
Let scar detect a maven installation and allow dependencies to be specified as maven coordinates:


dependencies:
- ../skorpios-desktop
- ../kryonet
- ../yamlbeans
- org.someorg.groupId:artifactId:version

I like mavens dependency management and repositories, but I absolutely hate it as a build tool.

#14

Well, maybe I’m being too hard on BeanShell. When I looked at it a few years ago, I ended up having to dig into the source. I wasn’t impressed by the organization of the source. (In comparison, Pnuts source is very nice.) Futher, I had a classloader problem and wasn’t impressed with the BeanShell author’s responses. Maybe I can dig it up… Sorry I only found it on a crappy site, but here is my problem post and someone saying it has been a problem for years:
http://osdir.com/ml/java.beanshell.user/2005-04/msg00022.html
My suggestion for a fix:
http://osdir.com/ml/java.beanshell.user/2005-04/msg00024.html
My suggestion for a fix, reiterated because Pat was not getting it:
http://osdir.com/ml/java.beanshell.user/2005-04/msg00030.html
There was no response after that. At any rate, I moved on to using Pnuts and have been very happy. Maybe I’ll look at BeanShell again, it has been a while.

Agreed. I will make the arguments available from the Scar class.

[quote]- Make project a global, imho the yaml file is the project, so why have to pass it to every target
[/quote]
Can’t be global as sometimes you need to build (or at least construct/query) a project in the middle of building a different project. I like keeping the utilities/targets completely separate from the project descriptor. It means the Scar utility methods are optional and the project can be highly customzied or even used to build other languages or do script-like tasks. I don’t think passing in a project when needed is too big of a deal, and is at least clear as to what is going on. The less magic, the better.

[quote]On a personal note:
Let scar detect a maven installation and allow dependencies to be specified as maven coordinates:


dependencies:
- ../skorpios-desktop
- ../kryonet
- ../yamlbeans
- org.someorg.groupId:artifactId:version

I like mavens dependency management and repositories, but I absolutely hate it as a build tool.
[/quote]
Totally agree, this would be a great addition. I imagine it would also be pretty easy to add a utility method that could push to a Maven repo.

#15

@Beanshell:
It’s dead man, BeanShell is dead!

At least I see no updates on the web page.
Sadly because I really like BeanShell and still use it today. Also Jboss comes with it.

#16

www.Janino.net is about 1000 times better than Beanshell! Faster, less bugs, better maintained. But for some reason less well known.

Btw good use of the javax.tools API in the Scar.compile method. Never seen it used until this project.

Re the dragon jnlp it launches so that’s cool but I get an error that the ‘Host discovery timed out.’

#17

IMO opinion this down to the name of the libraries, Beanshell sounds pretty cool and easy to remember, while janino is rather awkward and hard to remember. Guess we have another example of this with the names of the current java opengl binding.

@NateS the scar link in your signature is out of date as it points to the old link as well as the first post in this thread, might be a good idea to update or remove the temp scar page.

#18

Hi!

It’s Michael’s choice ;D

Maybe I will try Scar one day but I’m already happy with Ant and custom Ant tasks written in Java, JavaScript, Groovy, Jython, BeanShell… Do you plan to create an Eclipse plug-in for Scar?

#19

I hadn’t thought about it. I’ve never had the need for a custom Eclipse builder. I could see it being useful for some scenarios though, such as kicking off compile time bytecode manipulation (eg, JStackAlloc). So, to answer your question, probably not until I (or someone else interested in using Scar) have the need.