We treated .gems like source tarballs (they can't even express non-Ruby dependencies, Gem::Specification#requirements are treated as comments), and only used rubygems.org for newer versions of our dependencies to be packaged as .rpms for test and production deployments. I found that to be much more sane than bypassing the system package manager and smuggling random crap onto dev servers, much less production.