I think he says "the biggest fucking idiots" in the way some of us say, "aren't thinking of all the angles".

If folks are getting wrapped up in the insult, they're not reading his tone correctly. He says it lovingly, and even if you disagree with that kind of tone, it's disingenuous to say he's literally calling these people "fucking idiots".