Possible heap pollution... but what does it mean really?

I have this method:

    @SafeVarargs
    public final <T extends AABB> List<T> insideAABB(AABB container, boolean trimToCircle, List<T>... candidates) {
        ...
    }

Its tagged with @SafeVarargs to get rid of the compiler warning, but I’m not sure when I would would not be safe. And just how bad is a polluted heap anyway? From what I can see it only means ClassCastExceptions, which don’t have much of a lifespan in an average debugging cycle…