I consider that the browser's obligation. If I'm not specifying any font or background color, I cannot be responsible to add special media queries because someone chose a dark theme.
Thank you for bringing this up. I had been looking for an “easy” way to enable a Dark Mode on my website without overriding user-configured default colors and this seems to do it pretty well.
My page uses a minimalistic CSS and only changes the colors for syntax highlighting -- which is now slightly broken in the dark mode but probably still an improvement over the white background for those people who want the dark mode :)
My current solution is to switch the syntax highlighting to monochrome which should work fine for both modes. I love the colorful syntax highlighting, though. So maybe I will revisit this later to find out if I can do something along the lines of what you suggest.
One problem though is that the default link colors can be unreadable in dark mode. That's why I use @media (prefers-color-scheme: dark) to set a:link, a:visited, a:active colors.
i like the default (unstyled) link colors chrome uses, they seemingly do accessibility the best. on the other hand, safari's default link colors for dark mode are quite poor, e.g., a:link is dark blue, a:visited is purple
worth noting, browsers also treat the default dark mode background colors differently: chrome uses black, safari uses dark gray, and firefox uses dark navy
<meta name="color-scheme" content="light dark">