I don’t know why there aren’t full fledged computers in a GPU sized package. Just run windows on your GPU, Linux on your main cpu. There’s some challenges to overcome but I think it would be nice to be able to extend your arm PC with an x86 expansion, or extend your x86 PC with an ARM extension. Ditto for graphics, or other hardware accelerators
There are computers that size, but I guess you mean with a male PCIe plug on them?
If the card is running its own OS, what's the benefit of combining them that way? A high speed networking link will get you similar results and is flexible and cheap.
If the card isn't running its own OS, it's much easier to put all the CPU cores in the same socket. And the demand for both x86 and Arm cores at the same time is not very high.
You may be interested in SmartNICs/DPUs. They're essentially NICs with an on-board full computer. NVIDIA makes an ARM DPU line, and you can pick up the older gen BlueField 2's on eBay for about $400.