This is my experience with unit testing. Especialy, stopping other developers from breaking things is the point of it. Since the costs involved for the life of an application is mostly about maintenance and not it’s initial developpement, it’s imperative that developpers write code that test code even at the cost of having to write twice as much code :).
Absolutely true: unit tests are neither a rigorous nor complete specification, but if you think your program suffices for that, you go ahead and collect your Turing award for writing a better theorem prover than anything else out there. Unit tests are about recognizing that your specification is always going to have those cracks, even if you write it in something like Haskell, which has phenomenal power to specify things statically. Hell, even Agda code often has unit tests, and its type system makes Haskell look like PHP.
It’s true though that the more loosey-goosey the language, the more the community considers unit tests to be holy writ. Every time I see python and ruby programs checking the type of parameters, I shake my head and ask why they so strongly resist having a compiler make those kinds of tests for them.
That’s not a unit test so much as it is a regression test, which in my experience is half the time a functional test by nature (testing multiple things) and the other half some kind of corner case that I neglected in a unit test (that tests one thing – that is, a “unit”).
Opinions differ on unit tests, but I can’t conceive of any credible opposition to regression tests on known bugs.
I’m more into behavior driven development. This mainly applies if you’re trying to create a library i.e. shared code.
Writing test code that exercises the library will help ensure that the API is designed in a somewhat usable manner before it’s released.
I’ve extensively used JBehave (http://jbehave.org/) and indeed Behavior Driven Developpement is realy something to consider in library writting, I’ve got nice results with Web Service Interface testing too. The nice thing about this is that anyone can write a new test case can do it, even a non-programmer! 
BDD to me seems like extra annotations on unit tests. Might be useful for a QA guy writing the test skeletons that a dev fills out, but it doesn’t strike me as being particularly different from unit tests. Something that actually is different is QuickCheck, where the test system generates tests for you based on logical properties you assert about your code. It’s still in its infancy for languages like Java, which can’t express meaningful properties in the type system, but DBC-like annotations also work.
I’d say it complementary because it’s more about integration tests than unit testing. It’s very usefull for an analyst, who most of the time don’t write code but want to test the application, it’s often faster for him to write a test in plain english than retesting all cases one by one by point and click the interface each time developpers make changes.
Yah, I guess I’m colored by writing Specs2/ScalaCheck unit tests, which have a sort of BDD-styled structure to them but are otherwise still plain old unit tests. Frankly QA has no business writing unit tests, but they do come up with imaginative ways to break whole systems that demand a way for them to communicate the specific tests back to devs. I’m not sure jbehave is it, but it beats vague handwavey “write tests in a spreadsheet” nonsense at any rate.
I don’t like writing tests.
But what I even more dislike is haunting bugs longs after you think you’ve finished something.
Writing tests after the puppets started to dance is no fun.
We should look at tests as being a part of the implementation process. Tests offer a good, fast and isolated sandbox for new functionality, instead of launching the whole application over and over again.
I managed to write at least about 200 tests for my little Indie game which is not much considered the whole code base. But I am really glad to have them.
Relaxes the relationship between you and your codebase. ;D
There should be a new, cooler name for tests that could also be admired by the “testing-is-for-wimps-party”. ;D
I haven’t ever used them extensively, but generally I would agree with you.
However, if I were my friend who works on the computers for an airborne windmill (that will crash if has a bug), or if you’re writing the code for a nuclear missile, or anything like that… unit test the shit out of it.
Were I writing anything for nuclear, I wouldn’t be using Java in the first place but something with built in DBC.
Also… hunting bugs: I rarely seem to have to hunt bugs these days. Usually something goes wrong and I have a little trail of breadcrumbs left by a stacktrace to follow right back to what caused it. Failing that I pop in a couple of system out printlns (yes, really) or on occasion resort to the debugger in Eclipse. Bugs rarely take long to find or fix. I do have the luxury of having almost no-one else working on my code at the moment. Occasionally Chaz breaks it, but then again, he can’t really program
Cas
You use Specs2, you know about Haskell and Agda… May I ask what are you doing in a game dev forum?
Those technologies are at the opposite end of the spectrum from the game development world. Is there any intersection or application of those types of technologies to games?
I use JUnit for two kind of things (not necessarily being “strict” unit-tests):
- creating testbeds for something I have to experiment with.
- keeping others from breaking key functionality
since I have to do testbeds with mock-services for a lot of stuff anyway - because the real environments are unavailable, clunky or test-roundtrips to time-consuming - doing them in JUnit and throwing in some asserts in the end does not really hurt.
and keepin others from breaking key functionality once they take over is more or less mandatory. if you won’t do it with tests you would need a huge amount of knowledge transfer and coaching to ramp up your successor. this is a real time sink and if you fail you will accumulate a lot of legacy projects you have to take care of, which is very undesirable - at least at my current job profile.
having said that, doing strict unit testing and 90+% code coverage, is just a waste of time. you have to concentrate on the critical stuff and to quickly simulate an environment to save you time.
That tended to degenerate into arguments about what exactly was critical as I recall.
It still doesn’t change the fact that maintaining a unit test alongside the code it was meant to test basically doubles the development time, and from what I’ve seen so far the biggest use of unit tests appears to be “protecting my code from some other developer breaking it” which is beginning to sound patronising towards colleagues. On the very rare occasion someone has ever accidentally broken your code in the days before unit testing got trendy, had the issue not been resolved usually quite rapidly thereafter? Every time I add up the numbers the same answer comes out: you get and maintain more working code, in less time, without unit testing, in most types of projects.
Cas
I write business software that sees an internal audience of less than a couple dozen people tops, to zero at least (i.e. automated processes). It’s all in a range of languages, including lots of shell scripts, perl, python, java, and now scala. Other than the occasional flash of clever insights here and there, it’s a grindingly dull soul-sucking endeavor, so gaming is that escape fantasy of mine that I’m probably never going to seriously dive into. I also used to do a lot of hacking on MUDs (MOO mostly) which is about as close as I got to doing real game work, so something RPG-related is still what I have that itch to do.
Honestly, I’ve never used Agda, I only hear Haskellers talking about it. I love Haskell because it reminds me of how smart I am – that is, not smart enough to learn all of Haskell, at least not without cracking open my theory books. It’s stretching exercises for the brain. Even then I’m not sure what actual useful programs I would write in it.
Well, lucky for me (Har har) I’m going through a course right now talking about the various Testing Techniques. The point that’s been expounded upon by the professor, in regards to Unit Testing, is that it really shouldn’t be written by the people coding the chunks. It should be written by people who understand testing, people who haven’t touched the code in other ways, and should be used as a method for which the ‘correctness’ of a chunk of code can be gauged. Hopefully, the interfaces/contracts that each method needs to fulfill is well known and the testers can write the tests in parallel otherwise it will take a lot of extra time to get things done.
If it’s written correctly, it should be able to provide various benefits namely: Did the change that I just made cause this set of tests to fail? And to hopefully provide a bit more information than a stack trace does especially when it comes to side-effects. If it’s robust enough, it’ll be able to help you pin-point the error (Mistake in code) instead of just finding the failure (The chunk of code where the mistake actually caused an issue that we can see).
And then, the professor spends the next four lectures or so informing us that all testing strategies are sub-optimal, but that they’re better than the alternative. And especially that you have to pick your battles in regards to what you test. If it’s something that you can’t unit test reliably, don’t do unit testing on it!
An ideal world where you’ve got one person doing the specs, another doing the code, another doing the tests, and so on, where everyone sticks to their responsibilities without being hampered by the barriers between each other … it’s like that frictionless vacuum physicists need for their models to come out perfectly. Except it’s probably easier to create a frictionless vacuum.
Unit tests don’t measure correctness, proofs do.
well, if you expect expertise in programming, so unit tests are superfluous, I would expect expertise in finding out whats critical and whats not.
thats not my experience, if used sensible. I have seen a lot of unit tests for getters and setters in my life, though… :-/
I am talking about this:
- you or you team write a large non-trivial software system for a department of a company
- you get assigned otherwise
- someone else or another team takes over
I have seen enough sideeffects by seemingly trivial changes that it’s really a good idea to have the most complex or key functionality automatically tested. the new team has no idea how things work under the hood, so I consider giving them some way to be reassured that the system will work after their changes is just being polite, not patronising them…
No 
It depends on the project and how you use the tools you were given - that’s it. Automated (unit-)testing is a tool - it’s up to you, to make the best out of it.
As I said before, in any case you have create some environment to allow a quick rountrip for trying stuff out, doing it as a JUnit test is a no-brainer - you’ll set up something anyway.
And for testing critical stuff. That’s not only about the cost of writing and maintaining a software system, but also about preventing collateral damage, if the system fails. But this might greatly differ depending on the type of software system, the environment it is used in and the audience it’s using. I do most stuff for insurance companies, banks and business-to-business marketplaces, so there might be completely different circumstances.
While having your programs being tested by unbiased people is mandatory, letting them code unit tests is largely inefficient. Tests should be written while coding functionality. Thus, ideally, the benefit is not only a bunch of tests as side product but additionally faster development.
Furthermore, black box testing against given interfaces is likely to be incomplete, for anything non-trivial.
I used to think that someone else should write the unit tests but then I realised that unless it’s another programmer who is already working very closely on the same problem they’re not going to have any idea how to write a proper test for the code in question.
Cas