I use terminology today, which is also "fast." The major issue I run into with ghostty vs terminology is that ssh in ghostty is broken due to the terminfo thing. I don't fully understand why ghostty can't pretend to be a different terminal that remotes know about (for now), but I assume there's some reason.
Have you tried the suggestions in https://ghostty.org/docs/help/terminfo#ssh? I don't know what issue you may be experiencing but this solved my issue with using htop in an ssh session.
That is essentially the 2nd solution recommended. I think ghostty could do this itself as a hack. If the target binary is ssh (heuristic, but, ok), inject TERM=xterm-256color into env of the child process at launch.
I guess it isn't a huge deal to have every user to modify their ssh_config instead, but it's an ergonomic pain point for many new users.
I live in vim and after about 30 seconds of checking ghostty out I switched from iTerm2 to ghostty for good. No regerts.