> There's no technical enforcement of the policies in the file, it's up to the client to honor them.

That's incorrect. Cloudflare does in fact enforce this at a technical level. Cloudflare has been doing bot detection for years and can pretty reliably detect when bots are not following robots.txt and then block them.