The biggest issue I found is that X apps look like ass against the elegant Mac environment. Maybe that's mitigated somewhat by macOS 26's Android-ass looking UI, but the appearance/functionality clash between Mac and generic Unix was a major issue before.
I agree, I think the problem is the seamless integration, where it renders only the application against the macOS environment. I'd much prefer something more like the cocoa-way example where there's a window that has its own background, and the applications run inside that. Not sure if Xquartz supports running a compositor or windowing manager.
XQuartz used to support rooted mode. I played with an early version back in the PowerPC era, and ran a regular desktop with WindowMaker and everything, using software from MacPorts. It was kind of a "parallel universe", as XQuartz would take over the whole screen in rooted mode and you had to switch between it and the Mac desktop, but it looked and functioned like a typical Linux or Unix desktop of the early 2000s.
What issues are you seeing with it? I use it pretty heavily and have no complaints…