Only if you compiled your kernel with literally every module. If you compile your kernel with only the modules your system needs, there’s no such issue

which then made it difficult to upgrade your hardware.

Well yes, when you start customizing Gentoo it tends to make hardware changes more difficult. e.g. march=native makes CPU changes difficult, but it's still very common on Gentoo