You can have something IDE agnostic but still not be dependent on the ancient VT100 terminal protocol and rendering path.

(That said I do like being able to SSH in and run an agent that way. But there are other remote access modalities.)