Due to the way the mouse events are handled I don’t think it’s possible to implement this behaviour in an entirely reliable way.
As suggested above, you can add a MouseListener to your Frame, and listen for delivery of the mouseExited(…) event.
You can then reposition the mouse cursor using java.awt.Robot#mouseMove(int x, int y).
However, if a mouse click occurs while the mouse is outside the Frame - focus will be lost and everything will go tits. (break)
If you introduce a significant border between the Frame boundaries and the area in which the cursor is intended to loop, it will work better but never be fool-proof.
Extending this idea further, you could constrain the mouse cursor to the centre of the Frame(resetting the position after every mouseMoved/Dragged event), make the cursor invisible, and render a virtual cursor as part of your game loop. (if a game is what you are writing)
This approach will lose the UI benefit of a hardware cursor, but will make it extremely difficult for the cursor to skip outside your Frame.
Fundamentally AWT does not currently expose the necessary functionality to limit the range of mouse movement ( In Windows I believe you’d want to use the gdi ClipCursor function)
So the solutions suggested above are essentially hacks.