Ok… so i only recently got this working and im sure there is some optomization that can be done. but I promised on another thread to post a vid and source on this once it was working
This video walks through the theory and specifics of raycasting.
mCSvtUvlt5I
This is the code utils i created. all code is using LWJGL.
UnprojectUtil
http://pastebin.com/CLduy9gk
RayCastUtil
http://pastebin.com/AkdX4hba
there are three main steps in raycasting
Unproject the mouse coords for the 0 and 1 depth values
Calculate intersects with a plane for the surface you want to test against
Calculate the area of the surface (as a triangle) and then of the intersect point using the same triangle points… if the sum of the triangles - the original area of the full triangle == 0 (or < 0.00001) then you are inside the triangle.
EDIT updated to include video link
EDIT corrected typo
wVh0NMVP-uQ
Revised code
http://pastebin.com/XeVwn0UQ
UPDATED SAMPLE CODE.
This paste bin has my fully working Raycastingutil