I was confused on that front because its quite fast on Linux. Brief research on the subject suggested a few things.
Some folks have performance issues with VERY large libraries of 80,000 to hundreds of thousands of books which aren't experienced by people with more normal sized libraries of dozens to single digit thousands of books. There may be some room to optimize this case but to be fair this is an edge case.
On Windows antivirus can cause issues if the library directory isn't excluded. This is fairly out of scope and is just the general suck of using the Windows platform.
Mac users complained of general slowness. There may well be room for improvement in this area.
Sending to device may be slower than expected if sending involves either transmitting books over the wifi depending on the speed of the underlying connection or especially if sending requires conversion. It might be worthwhile in this case to enable conversion of books on import so as to obviate the need to convert many books at once later.