The distinction is that random access to the files need not be provided.
If I give your phone access to /usr/private_key.txt then the OS has total control. If I instead give you a way to sign messages then the OS has much weaker ability to control (obviously some amount if the device is connected and capable of signing).