Retroweaver is a free, open-source bytecode weaver that enables you to take advantage of the new Java 1.5 language features, while still retaining total binary compatability with 1.4 virtual machines. Retroweaver operates by transforming Java class files compiled by a 1.5 compiler into version 1.4 class files which can then be run on any 1.4 virtual machine.
God bless,
-Toby Reyelts
So there are no problems with 1.5 implicitly using methods or classes that don’t exist in 1.4?
(e.g. I figured StringBuilder would replace a lot of the implicit use of StringBuffer when handling concatination. I think they could do that if you choose to compile for 1.5 because they don’t allow a 1.4 target.)
[quote]So there are no problems with 1.5 implicitly using methods or classes that don’t exist in 1.4?
[/quote]
I specifically handle all of those situations. Like you said, StringBuilder gets used for concatentation. New JDK 1.5 valueOf() methods get used for autoboxing. I replace calls to these classes/methods with calls to Retroweaver’s runtime library.
What Retroweaver does NOT do, is handle calls to any general new API in 1.5. For example, you can’t make calls out to java.lang.management or java.lang.instrument, and expect to not get a NoClassDefFoundError when you run in a 1.4VM.
God bless,
-Toby Reyelts
Sweet!
I’ll have to check it out… if only Eclipse supported 1.5.
[quote]Sweet!
I’ll have to check it out… if only Eclipse supported 1.5.
[/quote]
Great! The more victims^H^H^H^H^H^H^Husers it has, the quicker I’ll be able to take care of any missed edge cases. Looking forward to your feedback.
God bless,
-Toby Reyelts
Hi swpalmer,
Just thought I’d let you know that there was a bug in the first release that meant that StringBuilder wasn’t handled correctly. The bug has been fixed in 1.0RC2, which was just released.
To everyone else, Retroweaver just got a thumbs up from Neal Gafter - Sun’s Java compiler writer: http://forum.java.sun.com/thread.jsp?forum=316&thread=503547&tstart=0&trange=15
God bless,
-Toby Reyelts
FYI to save you reading that whole discussion, the reason given for not allowing “-source 1.5 -version 1.4” is:
[quote]The problem is not and engineering problem of how to implement it …The problem is a legal one in that it violates the Java platform specifications. That is not something we would allow our licensees to do, and we should not therefore do it ourselves. If we allow the hybrid language then we will need a language specification, a conformance suite, and licensing terms for the hybrid language.
[/quote]
i.e. it’s a support issue. I suspect the only “easy” answer to this is that in hindsight they should have had a JLS 3 which would have been the hybrid 1.4/1.5, and made what is to be the JLS 3 into JLS 4.
But they didn’t do that, and perhaps it’s too late now.
It’s also probable that there isn’t the manpower to set up the conformance suite for not one but two new JLS’s (this is a considerable amount of effort!). The other problems he mentions are pretty trivial I suspect.
EDIT: and we’re one of those teams that would practically kill for this functionality right now 
- but can’t afford to jump to retroweaver until it’s been extensively tested (either by lots of other people or by us when we have the time!)
[quote]and we’re one of those teams that would practically kill for this functionality right now - but can’t afford to jump to retroweaver until it’s been extensively tested (either by lots of other people or by us when we have the time!)
[/quote]
Hey blah^3,
What qualifies as extensive testing? What makes Retroweaver any different from any of the other bytecode enhancing products out there - i.e. JDO, AspectJ, etc…
God bless,
-Toby Reyelts
Just thought I’d announce that 1.0RC4 has been released with new support for enums and Iterable.
I’d also thought I’d mention that I have one user who is successfully using Retroweaver in combination with Jet. Now Cas has no reason not to try out Retroweaver 
God bless,
-Toby Reyelts
FYI:
Firstly, we don’t use any of the other bytecode products, for similar reasons :). NB we don’t use AspectJ in particular because of in the current release they’ve removed functionality - one of those being that you can no longer use it as anything except a bytecode modifier. Any bug in their system = we go out of business (can’t compile our own source any more). This is an unacceptable risk - and a totally unnecessary one - unless we have a support contract with them that guarantees they’d fix said bugs ASAP (we’re talking on the order of 2 working weeks max).
For Retro, we’d need to create a suite of unit tests, especially including anything exotic we use that may have been under-tested by others - e.g. we’ve got loads of dynamic classloading stuff.
We don’t need to do it ourselves, just need to believe it’s been done; if we did it we’d publish the source and an explanation of what each test was doing so others could avoid the effort themselves. If/when it’s being used by one organization that’s guaranteed to do that (e.g. an IBM) or is being used by several organizations that between them have done it probably well enough to cover all cases, then it becomes something we can consider.
Basically, you’ve got to appreciate that we go out of business if we try to use retro and it fails a couple of months down the line. There’s no two ways about it. And it’s not just a question of “does it work for us?”. It has to work for us and all our customers, which is a much more stringent requirement, exercising a much wider set of use-cases.
If retro stops working, we either have to do no work for X months until 1.5.0 becomes available, or rewrite all our code to 1.4. Sun will just laugh at us for putting ourselves in a zero-contingency-plan position.
Would you bet your entire company on a new narrowly tested product without a support contract and without the expertise to fix it yourself?
OTOH for any personal project I were starting now I’d use a combination of 1.5 and retro 1.4 right away, because the risks are small enough that I can accept them. A personal project doesn’t have to be cancelled if I have to leave it 1-5 months for a working compiler. I can’t just give the entire staff a 5 month holiday if the same happens with a production system…
I’m working on building a “comprehensive” test suite into Retroweaver. It sounds like that may help allay your fears. I’d love to add any test suites you’d like to see in Retroweaver.
[quote]Basically, you’ve got to appreciate that we go out of business if we try to use retro and it fails a couple of months down the line.
[/quote]
Why would that be? If Retroweaver “fails”, it’s going to fail on some code you wrote, and you can choose to either a) wait for me to fix the bug, b) fix the bug yourself (it is open source after all), or c) just write that one class in the 1.4 language. There’s no “massive failure” scenario here.
[quote]It has to work for us and all our customers, which is a much more stringent requirement, exercising a much wider set of use-cases.
[/quote]
Why would that be? Are you planning on having all of your customers run Retroweaver? If you’re building a library, you can simply weave your library and release that like you would any other 1.4 library.
[quote]If retro stops working, we either have to do no work for X months until 1.5.0 becomes available, or rewrite all our code to 1.4.
[/quote]
I’ve already explained why this isn’t true at all.
[quote]Would you bet your entire company on a new narrowly tested product without a support contract and without the expertise to fix it yourself?
[/quote]
No, but that’s not the case here.
[quote]OTOH for any personal project I were starting now I’d use a combination of 1.5 and retro 1.4 right away, because the risks are small enough that I can accept them.
[/quote]
I gladly invite you to use Retroweaver on some small project. It might give you a better feeling for what it is doing and the “risks” involved.
God bless,
-Toby Reyelts
As soon as you get one, we’ll throw you some ideas 
based on whatever approach you’re using (and will contribute code if we’ve got spare time).
a) can’t wait.
b) being allowed to fix something is not the same as having the ability to do so.
c) that “one class” may have very many dependencies where the way in which it uses other classes is dependent upon retro. For instance, the co-variants example I cited today: it’s perfectly reasonable that we would have a class that uses co-variants to interact with 20 or more interfaces. If we have “just write that one class in 1.4” then we actually also have to rewrite perhaps 15 interfaces. Oh, and then we have to rewrite EVERY class that used those interfaces - and that quickly can become a LOT of classes. All to workaround a temporary problem!
Unfortunately for us our sub-systems are very tightly integrated internally (EDIT: this is slightly ironic considering the reason for this is usually to make them easier to replace piecemeal - but under certain assumptions that include never changing the program structure itself, which some of the 1.5 features do), so even when it’s just one class that gets stuck we have a high chance of effectively being blocked on the whole of that particular system.
We have support contracts with our customers. We have to fulfil these or face harsh penalties. In some cases, if we can’t deliver a system any more, or it stops working for an extended period because we can’t implement a critical fix, someone else’s game development stalls until we can deliver it. Many of the bugs we deal with are tiny yet critical issues with live systems - we usually cannot turn around and say “we can’t patch that for another few weeks, I’m afraid”.
I recall that linux adoption was massively slowed at one period because many developers thought that open-sourcing code was sufficient to make it usable by businesses, without realising that open-source code is usually worthless to someone who’s going to run their core on it - they need a much higher level of support than the mere possibility that they could theoretically fix it themself!
[quote]As soon as you get one, we’ll throw you some ideas based on whatever approach you’re using (and will contribute code if we’ve got spare time).
[/quote]
You can start right now. Look at the test folder of the Retroweaver download, to see the simple tests I perform now. Send me any source code you’re afraid Retroweaver won’t handle correctly. Describe any problem you’re afraid Retroweaver won’t handle correctly.
[quote]a) can’t wait.
[/quote]
This is the same problem that you’d have if you depend upon someone else’s 3rd party library. If you “can’t wait”, pay me money, and I’ll guarantee you get the same response time you do for the product you paid several thousands of dollars for.
[quote]b) being allowed to fix something is not the same as having the ability to do so.
[/quote]
True, but it’s an option you don’t have at all with a commercial product.
[quote]c) that “one class” may have very many dependencies where the way in which it uses other classes is dependent upon retro.
[/quote]
Red herring. I don’t see somebody developing a new class, that, when first introduced, has hundreds of dependencies on it. Even if this were the case, you could easily have the same problem if somebody’s 3rd party library fails. (I.E. you return an instance of some class from a method call, all your other classes use that class, then you learn that that class is all busted, and you have to replace it with some totally dissimilar class. Now all your classes are busted).
[quote]We have support contracts with our customers. We have to fulfil these or face harsh penalties. In some cases, if we can’t deliver a system any more, or it stops working for an extended period because we can’t implement a critical fix, someone else’s game development stalls until we can deliver it.
[/quote]
How does this differ for Retroweaver than any other 3rd party product you use?
[quote]they need a much higher level of support than the mere possibility that they could theoretically fix it themself!
[/quote]
If you want paid support, it’s available. If you want access to source code - it’s free. That is far above and beyond what you’ll get with a commercial product. Btw, I often get asked if I provide paid support for Jace (I do), but hardly no one takes advantage of it. Turns out it’s mostly a security blanket.
God bless,
-Toby Reyelts
retroweaver is hot stuff! I am going to include it in our currently 1.5-only project here at java.net.
I didn’t take a closer look at retroweaver atm but is an easy integration with ant possible? A specialized task that lets me adding 1.4-aware targets?
— EDIT:
Ok i have seen it has no special ant task atm but this won’t be a problem. If I find the time i write one and send it to you.
another feature i want to see is package conversion. meaning the source classes are taken from a java-package and the result creates a new one.
nice example! look at this: “BTW -source 1.4 -target 1.4 compiles covariant return types for interface implementations just fine althought it is not part of 1.4 language specs (do not remove it please even if it is a bug :-)” taken from [url]http://forum.java.sun.com/thread.jsp?forum=316&thread=503547&tstart=0&trange=15
:PEDIT: I meant “forced to use 1.5” - but if I have to do that, I can’t resist the temptation to use 1.5 features ;). Thank god for RW!
Well, I’m being forced to use retroweaver now - gun against the head: jdk-1.4 java compiler is f***ed beyond all recognition (on a certain set of sourcefiles, for apparently no particular reason, I’ve accidentally discovered a bug in the javac compiler which CRASHES the compiler with a 100Mb memory leak [on source files < 100k]. Fortunately, 1.5 beta doesn’t have the same bug, so I can use 1.5 from now on to compile ).
So…quick comment: the download link is hard to find!
Also, the commandline the docs say you need to use is evil. Couldn’t you:
?
Also, +1 to the RFE to work on something more high-level than classes. I don’t actually have classfiles (why would I? They just make a mess…) except in a temporary chroot filesystem en-route to becoming JAR’s - the output from my compiler is JAR’s (c.f. vestasys.org) which is how javac might have worked itself if JAR’s had been invented 5 years sooner </random supposition>.
I would prefer not to mess with the build system (which is complex enough as it is) and to be able to apply retroweaving as a post-build step - but that’s only possible if you can point it at a JAR file and have it spit out a transformed version.
Obviously, I’ll just go ahead and manually un-JAR, weave, and re-jar for now. But it would be nice to be able to do just:
java -jar weaver.jar myGame.jar
;D
[quote]Thank god for RW!
[/quote]
;D
[quote]Fortunately, 1.5 beta doesn’t have the same bug, so I can use 1.5 from now on to compile.
[/quote]
That’s a novel reason to use Retroweaver.
[quote]So…quick comment: the download link is hard to find!
[/quote]
I’ll change that.
[quote]Couldn’t you: …
[/quote]
Probably. I was more interested in getting Retroweaver out there, working correctly, then making it insanely simple to use.
[quote]Also, +1 to the RFE to work on something more high-level than classes.
[/quote]
Umm… what RFE?
[quote]But it would be nice to be able to do just:
java -jar weaver.jar myGame.jar
[/quote]
Thanks for the input. I’ll look at making that happen, now that I’ve got the bigger stuff out of the way (a developer’s guide, etc…)
God bless,
-Toby Reyelts
I just wanted to let everyone know that I’ve released a new version of Retroweaver with a new GUI, a new Developer’s Guide, a new executable jar, and several bug fixes. I’ve also added a download link directly on the home page, so you can’t miss it. So, blah^3, the only thing missing from your requests is that I still don’t operate on jar files. Sorry.
Lots of people have been asking for IDE plugins for Retroweaver (IDEA, Eclipse), so that’ll probably be the direction of the next release.
God bless,
-Toby Reyelts
[quote]Lots of people have been asking for IDE plugins for Retroweaver (IDEA, Eclipse), so that’ll probably be the direction of the next release.
[/quote]
I noticed that Borland’s free Foundation JBuilder v11 (aka 2005) also features something like your Retroweaver: you can compile your Java 1.5 sources against JVM 1.4 and it works like charm.
Is there some link between your Retroweaver and theirs, or did they just re-invent the wheel?
It’s possible, but I have no idea. (People do have a tendency to ship my code without any sort of acknowledgment).
I know that CodeGuide also does this kind of thing, but, the last time I checked, I found their implementation to be lacking features and buggy. You might want to look thoroughly at the JBuilder documentation and run some simple test cases. For example, what happens if you implement java.lang.Iterable or subclass java.lang.Enum and try to run that on a 1.4 VM? Does their autoboxing implementation on 1.4 perform the 1.5 mandated caching? Etc…
God bless,
-Toby Reyelts