For the mostly high-level C code that I write, I find that compiler optimisations give typically give speed-ups in the 1.5x - 3x range. A lot of code is bound by external bottlenecks that would completely mask such a speedup anyway.

For really performance-oriented code you probably want to drop to SIMD first, and play with compiler optimisations second.