The manufacturers do their best to protect the users, but when it comes to criminal investigations there are tools[0] that can often make access to smartphone (or computers) data possible.
These tools are often prohibitively expensive and/or only licensed to LEO's (Law Enforcement Officers) and cannot unlock/access "everything", so there is no real "privacy risk" connected to them unless you are charged with a criminal offence and/or arrested.
To give you an example there are cases where an iPhone can be unlocked by two (AFAIK) different tools, one is made by Cellebrite that charges an awful amount of money for each unlock, and the other is Graykey that wants as well a lot of money but you can buy the "unlimited" option:
both are only given to authorized investigators (Police/Government) only.
Of course it is possible - in theory - that someone malicious manages to get their hands on one of them and then proceeds to steal the phone from you, and then can afford to spend anything between 3,000 and 10,000 US$ to unlock it and access your data, but I find it improbable.
[0] until the manufacturer patches (if patchable) the vulnerability, and then the race starts to find a new one