But they key is right there in the beginning; photons are NOT small billiard balls moving along ballistic trajectories.
"Wave-particle duality" doesn't mean that you can think of flicking a light switch as alternatively an electromagnetic field propagating from the light bulb, or that the light bulb shoots out a stream of tiny billiard balls.
No, there's only the electromagnetic field. However, if you squint closely enough you see that the energy levels of the field are quantized (aka photons).
Now, back to the double slit experiment. So a wave propagates from the source, through the slits, interferes constructively or destructively, and then hits the detector screen. However, due to the quantum nature of, well, everything, the "hits the detector" interaction is quantized. The field "deposits" quantas of energy (photons) to the screen in a localized interaction (say, a photon causes an electron in the screen to jump to a higher energy level). If the detection apparatus is sensitive enough you'll see that single interaction as a dot on the screen, rather than a faint interference pattern (which you'll eventually start to see if you repeat the experiment long enough).
So: Lets forget about the "wave-particle duality" already. There's only fields. Fields which interact in a quantized fashion.