Howto: Open Sourcing a Project

Miscellaneous Topics
#1

Hopefully not to many people are coming here looking for the answer on how to open source a project, cause well, I’m not going to be giving that answer… I’m going to be begging for it.

I’ve been working on a .x object loader (the directx format), and I currently have support for the most basic stuff, things like texture coords and all that good stuff you’d want from any static model, I intend to include support for skeleton code, though I haven’t yet since it’s not useful for my current game project. I’m planning to post the code (along with an example usage) in the shared code section in a few days after I put a little more work into it, but after that I’m feeling tempted to try to maintain the code as open source. I’m pretty confident that a few people here will find my code useful (that doesn’t mean they won’t make fun of my code, but as far as I’m aware no .x loader is available yet). I currently don’t have plans to support a large variety of the keywords possible in a .x file (they are ignored if I don’t know what to do with them) and I am not planning to support compressed or binary files (for now, though obviously a binary version of a .x text file would be a LOT smaller). I also don’t expect to be building in much of a safegaurd against corrupted files/files that aren’t formatted correctly, we’re talking a bare bones object loader here (and a data structure to hold the info, which jogl users can use… should my code make it into any of the scenegraphs being worked on I’m pretty sure my data structure would get tossed out, as it just mimicks the data structure of the .x files).

My question is, what do I need to know to manage an open source project reasonably well, I have no experience working in any kind of team environment (at least not where any other programmers would be making updates to my work), what tools should I check out, or articles should I read, to keep it from being a nightmare to maintain? I suspect that I’m going to get lucky, have at least one poor fool submit a patch of some sort to me that does something really awesome, and I’d just let the patch sit on my hard drive for months without applying the code. Anyway, hopefully in a few days I’ll have it ready enough to post.

#2

Sourceforge have a pretty good selection of tools already available (CVS for version control, mailing lists, forums and so on). I’d recommend at least starting off at somewhere like sourceforge (or one of the other similar OSS hosting sites), because at least you don’t have to worry about the nitty gritty of setting everything up yourself (unless of course you’re a sucker for punishment).

That said, we use Subversion at work, so don’t use sourceforge for anything more than a release repository, and for the mailing lists.

#3

Yeah, SF is a good place to start - but their CVS is legendarily (no, really) crap; don’t touch it with a bargepole. IME it’s better to distribute your source as source, and get people to send you diff / patch files and you apply them directly, especially if you’re relatively new to managing CVS (I don’t thinkg sf.net’s CVS has ever worked for more than 6 months consecutively without breaking down).

It’s definitely worth learning about diff and patch if you don’t already know them - very simple tools that let someone send you a single text file which you “run” and it will patch your source to match theirs. Even if you don’t use them, and choose to use automated CVS (yuk) for everything, they’re still worth knowing how to use.

But…IMHO I would rather headbutt a brick wall repeatedly until I slowly slipped into unconsciousness than use CVS on any project ever again. It is, in a word, crap. I know a lot of people get upset when I say that, but it simply doesn’t have any redeeming features - everything else you can get (many of which are open-source, many more are free) is better in at least one way, and usually better in many ways. There’s no excuse for CVS; like Windows 3.1 it’s a complete waste of time in this day and age.

#4

I assume that I’ll be making updates to my code, and that someone who might have started working on an update a month ago and finally sends it in, are you saying there are tools that can handle that kind of situation? Another one is if I modify a peice of code that someone elses modifies (it could happen especially early on). I’m guessing this is the kind of thing that makes open source a nightmare because now I have to understand someone elses code to implement it with my own.

The real life situation that’d probably cause this, someone downloads the source and says holy crap, this is the worst source I’ve ever seen, they refactor it, they clean it up, they make it so that there are accessor methods instead of directly accessing variables (which right now is the case, I wasn’t 100% sure the data strucutre until I had things done so I didn’t want to bother with setting up constructors and having to change them as I worked… and really, for something like this direct access is just as well I think). While they completely re-arranged the code most likely for the better, I added in code to better check the data as it’s loaded to prevent any errors from floating up and possibly outside of the package (like null pointers or StringTokenizer exceptions which will pretty much halt the JVM if they aren’t caught. Both of these changes would heavily modify the same section of code, is it just a matter of deciding which version is the easiest to implement the other changes into?

I’m off to research the stuff you guys mentioned and do some searchin for more info.

#5

[quote]I assume that I’ll be making updates to my code, and that someone who might have started working on an update a month ago and finally sends it in, are you saying there are tools that can handle that kind of situation?
[/quote]
Most of the popular version control systems handle multiple updates. They basically do their best to merge the two changes together but if modifications cross over line boundaries they throw out collision files containing the bits they can’t figure out.

[quote]Yeah, SF is a good place to start - but their CVS is legendarily (no, really) crap; don’t touch it with a bargepole.

…

But…IMHO I would rather headbutt a brick wall repeatedly until I slowly slipped into unconsciousness than use CVS on any project ever again.
[/quote]
Couldn’t agree more about CVS – which is why I never used SF’s CVS. I’ve had some insanely annoying problems with CVS in the past… which is why I daily thank [insert-your-personal-deity-here] for Subversion.

#6

[quote]I assume that I’ll be making updates to my code, and that someone who might have started working on an update a month ago and finally sends it in, are you saying there are tools that can handle that kind of situation?
[/quote]
This is usually called “conflict resolution” (google; the algorithms used are documented if you’re interested).

One of my reasons for disliking CVS is that it tries to do this automatically - the whole system is designed on the basis that this is possible and works - and has bugs in its code which have screwed up projects I’ve worked on in the past. These may have been fixed by now - last time I checked, they hadn’t.

CVS is designed for archaic C programmers, not java programmers. It’s designed for people with massive source files, not small well-designed classes. CVS has few benefits to modern OOP developers because it was designed for a different way of working.

If that ever happens to you then you’ll probably just accept their changes without question ;D. On the whole, most SF project owners secretly wish for this every day, and it never ever happens.

Look at it this way: if your code is so impenetrable, you will turn away most of the people who would have helped. So, in practice, you end up having to make those kinds of changes yourself, in order to attract other people to join your project. Not always, but nearly always. so I wouldn’t worry too much!

Usually, most developers won’t embark on any such changes without discussing it with you first, in case you’re already doing it yourself. Or if you disappear for 6 months they may just go “Sod it, I’ll do it anyway”.

#7

Kind of what I figure, the reality is it’s not likely that anyone will submit any patches to me, though I believe there is a good chance that the code could serve as a base for loading into some of the current scene graphs so the code is likely to be useful, just not community inspiring :stuck_out_tongue:

#8

IMHO a lone, feature incomplete, .x loader is not a good basis for an opensource project ie not likely to attract contributors or users. But you can be like me and do it anyway, just for the learning experience…

#9

[quote]Yeah, SF is a good place to start - but their CVS is legendarily (no, really) crap; don’t touch it with a bargepole.
[/quote]
Hmmm… I’ve used CVS for years and had no problems at all, and I’ve used CVS from SF plenty of times with no issues. What’s your beef with CVS and CVS @ SF?

#10

??? AFAIK Subversion is CVS with only very minor improvements (move retains history, binary diffs, a few other minor differences)

How can you hate CVS and love SV which is very nearly exactly the same thing?

#11

[quote]IMHO a lone, feature incomplete, .x loader is not a good basis for an opensource project ie not likely to attract contributors or users. But you can be like me and do it anyway, just for the learning experience…
[/quote]
A complete feature rich .x loader wouldn’t attract any contributors either (at least not sane or useful ones). Something written to be a utility for 3d java programmers is also not likely to get a lot of attention either, but as is there aren’t a lot of tools available, and even incomplete a .x loader is useful, it’s not incomplete because it’s not possible for me to complete it, it’ll be incomplete because it’ll have all the features I require and I do have to do other things like build games (the reason I put together the .x loader).

I’m pretty sure I personally won’t go beyond supporting animation (which is one step further than I currently expect to need for any of my planned projects). I think probably the hardest thing about the .x file format to support would be custom templates, it’d be interesting as a challenge to support it, but if I was going to I’d probably be better off structuring the entire package differently (so the standard templates would be handled the same as custom templates instead of hard coded).

From what I’ve seen and because I don’t plan to promote the project itself beyond these forums so it’s unlikely any contributions would happen other than what users here contribute. Anyway, it’s looking like I won’t have the source ready to post until after this weekend, mostly because the example program needs cleaned up and probably commented.

#12

HiI feel i must pitch in and defend CVS a little bit :slight_smile:
I have used CVS in many projects and even tough it is most certainly not perfect, it gets the job done. CVS tries to merge different versions automatically and I dont see a problem with that, most version tools do this and if it dont work(i.e conflict) you have markers in your file that tells you where the conflicts occured so you can easily resolve them before commiting the changes. I don’t see any difference using CVS for C or Java projects either.
On Topic: I agree SF is a good place to start and although I dont have my own project there, I have used their cvs many times with no problems.

// Gregof

#13

Erm… easily? Put it this way, the bugs I’ve experienced with SVN haven’t resulted in a massive loss of code. It seems easier to maintain the repository. SVN revisions are an entire tree rather than per file. Copying and deleting works better, at least in my experience. Conflicts are easier to track in a large repository. It uses apache as the server (and apache’s authentication mechanism).

In short, what’s not to like?

#14

[quote]HiI feel i must pitch in and defend CVS a little bit :slight_smile:
I have used CVS in many projects and even tough it is most certainly not perfect, it gets the job done.
[/quote]
So does notepad, but we (almost) all use eclipse, IDEA, jbuilder etc instead - they are worlds apart in productivity.

That’s because you are too ignorant of what CVS is actually doing, and don’t realise the sheer crapness of the thing you are using; you have been working “fingers crossed” and been lucky so far, but one of these days it will blow up in your face :frowning: (I speak from bitter experience). For instance, CVS has for many years (and it was known about for years and NOT fixed) had a fatal flaw because of the sheer stupidity of the authors who were too ****ing dumb to understand the basics of multi-users systems and “synchronization” and THERE WAS NO LOCKING so that if two people checked in at once ONE PERSON’S STUFF GOT CORRUPTED. This is not a “yeah, but in reality it doesn’t happen”; it does happen, it has happened, and I’ve met at least 3 people from different projects who’ve experienced it first hand! Really, if you think about it, it’s actually highly likely that some checkins will be simultaneous, especially if you use CVS for an office system.

Actually, no - most tools make it impossible by using checkout-time-locking (by a narrow margin; almost as many tools these days do checkin-time-locking instead or as an optional alternative), which in a modern OOP project is usually better than lockless control, because there are the few advantages to lockless tend to fade away and vanish on such projects. In non-OOP projects, with large monolithic source files, lockless control has its major advantages (but locking still has advantages of its own, and choosing one or the other becomes a decision with no “perfect” option).

CVS was designed to get around one single problem:

  • person A checks out file BLAH.c
  • (most tools would lock BLAH.c preventing anyone else from changing it)
  • person A starts editing lines 1500-1600
  • person B wants to edit a small change to line 12
  • …person B has to sit around drinking coffee waiting for A to finish

In a good OOP project, it’s usually very difficult for two people to make simultaneous non-conflicting changes to the same source because OOP implicitly splits up those large source files into a much larger quantity of much smaller source files - this is part of the concept of Objects: the refactoring into smaller pieces that are both easier to re-use and also better isolated from changes elsewhere in the code.

Unless they don’t keep their news, you should be able to go back trhough the archives of SF front-page news and find the multitude of “CVS server is not working again” reports that often last a fortnight or more.

#15

Exactly. CVS is the only source code versioning system I’ve ever encountered where the authors did not care about whether they corrupted your source code. All the other tools make sure - no matter what happens, no matter what breaks or crashes - that they don’t damage existing code.

That, in itself, is reason enough to scare me away from CVS forever.

#16

I just read an article about svn in the latest issue of c’t (a german computer mag). Well, it seems to be really nice. Unfortunately it doesn’t work on win9x (win98se in my case) :-/

#17

[quote]Unfortunately it doesn’t work on win9x
[/quote]
Nothing does :stuck_out_tongue:

#18

AH HA! I’m not the last person on earth using windows 98.

#19

I believe that there’s a version of Solitaire that works on Win98.

Oh yeah, and I’ve also heard that a computer running Win98 makes a really good door stop…

;D

#20

Pf :stuck_out_tongue:

It’s running for almost 5 years now (yes, it’s that first install I did on this machine). And frankly I don’t know anyone who managed something like that with *nix, 2k or whatsoever. Oh and yes, I use this machine… 10-24 hours per day and I installed and uninstalled lot’s of software in that timespan.

So… yea… you guys can kiss my shiny metal arse :-* ;D

Well, it’s really annoying that I can’t run that stuff. The other machines aren’t 24/7 up and the switch has only 5 ports, which are all in use. So… all options are rather expensive :-/