Forget non-free CUDA runtime, there are larger problems with the CUDA toolchain with respect to free software. The actual instructions executed by Nvidia's chips are pretty much undocumented. Nvidia provides great documentation for PTX, which is essentially assembly for a virtual GPU. All tools to convert PTX to SASS (the actual instructions ran on GPU) are proprietary and undocumented. Nvidia does this because it gives them much more freedom to modify the SASS language as well as the chips themselves. The existence of standards-free SASS code (and no open source compiler that can target SASS) makes truly open source GPU computing on Nvidia hardware impossible.