It's "on" enough to detect the activity needed to wake it back up. But will a powered-off phone still be pinging cell towers or making WiFi requests?

Yes, that's how Find My works.

If you cover your phone with an antielectrostatic bag it can't communicate; that is a Faraday cage.

Since people around you will think you are also wearing a tinfoil hard, you had better stick to the phones with hardware switches as sibling comment mentions

Most of those bags are total BS

Seems easy to verify with an iPhone. Put it in the bag, transport the phone, look it up on Find My.

Even easier, put any phone in the bag, call it. Does it ring? IME, yes.

Use one of the few phones with hardware kill switches or removable batteries.