This should be easy in Swing. The editor for the TableLayout DSL (which is now obsolete (the DSL is obsolete, TableLayout is still very much alive :))) does it using RSyntaxTextArea:
http://table-layout.googlecode.com/svn/wiki/jws/editor.jnlp
After an edit it tries to compile, catches any exception, determines the line and column for the error, and applies red squiggly underlines. RSyntaxTextArea makes error underlines, syntax highlighting, code completion, auto indent, curly brace matching, line wrap, etc very easy. Also the author is very responsive to requests. Source for the editor is here:
https://code.google.com/p/table-layout/source/browse/branches/v1/tablelayout-editor/src/com/esotericsoftware/tablelayout/swing/TableLayoutEditor.java#52
