Swap memory use is worse than actual memory use. If it just reserving a big block and not using it then fine whatever but if it's using a lot of swap that means ram is probably bottlenecking you since swap is slow.
It's of course slower than real RAM, but Apple are using pretty fast SSDs and possibly a novel model of swapping, making it effectively pretty good (certainly not perfect!).
