The argument that is made is that because you can entirely abolish a whole slew of agencies and other programs (welfare etc.) it's not as simple as just looking at current budgets, you need to factor in the savings you get from axing the agencies.

To be clear I'm not saying that it's viable, I'm just saying that your calculation is not the one proponents of the idea are using.