I would go even further: websites should not be the ultimate decider on any style-related decision. Including colors, font faces, text size, margins, padding, content width, and any other UI elements. Website should be able to suggest these things, but browsers should default to the user’s preference or the system’s configuration. The user should have the final say on how the User Agent displays HTML content.
Browsers have ceded far too much control over these things to websites, to the point where we are even debating whether they should style such fundamental OS controls as scroll bars!
It's relatively easy to force your custom stylesheets to any website. Whether it will work at all is another question.
I believe what you suggest could work only with semantic HTML without any advanced styling - no floating elements, no flexbox/grid, no absolute/relative positioning etc. This then means return to static text based websites which is fine for many use cases, but not for all.
Ideally, rich web apps which actually need all the CSS/APIs could use them, while the text-based sites would keep custom styling/JS to minimum (which would allow reasonable user-defined styling). But it's impossible to force this dichotomy in practice.
Overriding all css is so popular a concept that no Firefox plugin to do that has more than 100k downloads (and firefox users are probably the most likely to fidget with things like that).
A radical idea - people should be able to create whatever they want in whatever colours and fonts they want and have it shown to viewers the way they want, at least unless the viewer explicitly objects.
Browsers have ceded far too much control over these things to websites, to the point where we are even debating whether they should style such fundamental OS controls as scroll bars!