If you leave US mainland customs zone and go to saipan or Guam or something it will probably slow you down and they might claim you need it, even though you don't. Also Mexico never changed their law like US did, iirc you can still enter Mexico with a birth certificate.
If you're near a port of entry, you actually can be stopped at any time to be asked for proof of citizenship/visa/etc. See also https://www.aclu.org/know-your-rights/border-zone There are plenty of border patrol stops along highways that never cross a border where you'll be routinely stopped.
In practice, that means "near the border with Mexico" where there are border patrol stations along highways well inside the US that do routine stops, but technically it applies to any port city as well.
_Technically_, if you're a US citizen, you're not required to have proof of citizenship on you. But a drivers license in the US does not show that you are a citizen, and they absolutely can stop you from leaving (at least long enough to mess up your day plans) if you cannot actually prove citizenship in practice. It's just a _lot_ easier to have a passport on you. Passport cards exist mostly for this purpose.
E.g. I lived in Texas for a long time and have a passport card for that specific reason. Driving down to the beach or just going on random trips involves passing through border control stations even though you never leave the US or even get particularly near the border. Your chances of just going to visit a friend in south Texas and winding up needing to pass through a border control checkpoint due to unplanned road closures/etc are pretty high, so it's best to have a card in your wallet at all times that counts. Most remote highways have them as soon as you get back near a town. Most of the time, they'll wave you through pretty quickly, but it's just a lot faster process if you have a passport card and not only a drivers license.
Side note: A friend of mine was a park ranger at Carlsbad Caverns. He picked me up at the El Paso airport and we were driving to Carlsbad. We went through a border control checkpoint. They threatened to arrest him on the spot because he had his park service uniform hanging up in the back, but no employee ID or other proof that he actually was a ranger. They said he was impersonating a federal officer. They made a big deal of it and made us wait while they called the park to verify he was employed (and that didn't work because it was about 8pm). We eventually got let go with a warning after he found random online photos of him giving cave tours and convinced them that he couldn't possibly have staged a ton of random folks on the internet to have him in the background of photos.... But still, it was kinda nuts...
I think people often think something they did or didn't do lead them to be fucked with. Having gone through the border and checkpoint many many times with absolutely wild variation despite doing same stuff, my observation it is 99% whether the officer is bored or sadistic and 1% the totality of the situation.
Slight correction: "enhanced" driver licenses are citizen-only, and this is actually checked during issue, so they can also serve as proof of citizenship.
To clarify some of the confusion in this thread, a REAL ID driver's license is completely different from an enhanced driver's license. A REAL ID meets federal security standards and is available in all states. An enhanced driver's license proves U.S. citizenship and includes RFID but is only available in five states.
Enhanced driver licenses, not regular ones. These come with a whole lot of federal strings attached, and exist pretty much entirely for the purpose of traveling across the border without a passport. As far as I know, the feds require citizenship checks for any participating state.
Now, in some states, regular driver licenses have a citizen or non-citizen marker on them, but that's a whole separate thing (and AFAIK the feds don't consider them sufficient proof in any case).
