Gfx cards hardware accelerate matrix operations…or m i wrong?
How can we benefit from this option using java? Are there any special data structures within jogl maybe?
there are no such types like matrix etc, but you could program much of your engine in gpu using vertex sheaders for animation , pixel sheaders etc. Generally hotspot is quite fast for float operations and the latest version in mustang can make optimzed code for sse/sse2 instructions so wait for mustang, another libraries are not required. But of course such operations like glRotate etc probably are optimized by gpu driver so we can call it hardware acceleration 
Well you are right as a general idea, but as said above it isnt really a matrix multiplyer, its something more flexible.
If you use JOGL, Java3D or any other hardware accleeratesd 3D API then you are getting the vaule of this in your graphics already.
If you want to try to use the GPU (graphics processor unit) to do other kinds of calculations then you woudl write special shader-programs and shove them down with JOGL.
There is an entire website dedicated to such “other uses” of GPUs called www.gpgpu.org