I had another thought about the control issue today. What might be a good idea is to allow cursor keys to rotate the row or column that the mouse is hovering over. This allows you to keep your eyes on the area you are manipulating, and easily return your eyes to the action if you do need to look away, but still manipulate the rows quickly and easily.
Any thoughts about a control system like that? I personally think it will allow people to play a lot faster and more accurately.