Using a public key mechanism means you can have a system where there is literally no interface that allows you to extract the sensitive parts. A very secure cookie jar still requires you to take the secrets out of it to use them.