C# and Java are very similar as a language, C# is in fact much more like java than it is like C.
Both are OO languages, with single class inheritence/multiple inheritence through interfaces, both run on a VM, both have GC, etc.etc.
As for performance, C# and Java are comparable. As a language, while I see some advantages of C#, I still prefer Java as I feel it’s cleaner and simpler. C# seems to me less focused than Java, as if it can’t decide if it wants to please C++ developers, Java developers or VB developers.
I don’t know much about XNA, so I can’t really comment on that.
Pros of LWJGL I suppose it’s that it’s open source with a very liberal license, runs on Windows, Mac and Linux.
One big advantage of developing in java is that there are excellent free IDE’s like Eclipse and NetBeans which both are really much better than MS Visual Studio .NET Especially compared to C#, where you really have to buy some plugin (iirc called C sharper or something ReSharper).
A plus of C#/XNA might be that it has a big company behind it that really actively support games development. OTOH, the java games development community is quite active, and some great stuff has come from that, like LWJGL, jME, Xith3D, etc.
In the end, I suppose it depends on your goals:
If you’re a big company and want to target Windows, Windows, XBOX, and XBOX, C# might be your friend. I think targetting XBOX as a small independant developer is not so easy and it will cost, so you will probably target just Windows if you go indie.
With Java, at least you also get Linux and Mac as a bonus, and Mac is really a good market for indies! Java can even run on a PS3, but you won’t get hardware accellerated graphics, so fast 3D games are not really viable there. Who knows, that might change some time in the future…