As you are lonely in a cave and try to find your way out I think this game might work well with atmospheric background music and sounds. Not too much rythm and melodies. Keep it organic and the tone should be more dark than nice
I just looked a bit on soundcloud and found a nice example. https://soundcloud.com/cryo-chamber/dark-ambient
Listen to the parts from ~15:00-23:00 or ~ 33:00-39:00. This music gives me a very eerie feeling and I would find something similar very fitting for the game.
Also, I would add a lot of random sound effects to it. Sounds of dripping water, howling wind, bat sounds etc. what ever could fit in the cave scenario… Don’t overdo it and make sure that it fits in (play around with the volume, add reverb to it etc.) Don’t try to use repeating patterns.
About the rotation. I played a game that has been made for ludum dare 29 that has a similar mechanic, where you rotate the screen and change the gravity with that. The game is a bit different but I think the rotation works quite well there and the cut is not as harsh as in your version. You can check that out and see if maybe something similar would fit for your game.
http://www.ludumdare.com/compo/ludum-dare-29/?action=preview&uid=25122