Not the same person, but I first try and figure out an API that allows me to not panic in the first place.

Panics are a runtime memory safe way to encode an invariant, but I will generally prefer a compile time invariant if possible and not too cumbersome.

However, yes I will panic if I'm not already using unsafe and I can clearly prove the invariant I'm working with.