If you can devise a tool that can detect AI generated content, you can use it to filter data. But the harsh truth is that "gold standard" training data is from before 2022 or whenever the cutoff was.
And even that needs to be curated because before AI tools there was bot content filling up the internet.
...and even without bots, a lot of human authored content are low value, poorly written, etc.
There are (probably) companies out there whose business is to create, curate and improve training sets.
And if there's a way to detect that content is AI generated, then there's demand to generate content that seems more human. And we're already at a point where most people have been tricked into believing AI content was real at some point and never even realized it. It'll only get worse.
Probably the only real way to validate content is real is building a validation system into devices. Confirm when a photo is taken and send an ID to a server, then when photos are shared, its ID is compared to the image on the camera/phone manufacturer's server. For text, validate every little key press. And there are still ways to game these systems, but I would not be surprised if they're introduced to mitigate AI diffusing everywhere.
And even that needs to be curated because before AI tools there was bot content filling up the internet.
...and even without bots, a lot of human authored content are low value, poorly written, etc.
There are (probably) companies out there whose business is to create, curate and improve training sets.