Looks like the actual threat is that it's hard to get currently known chemical weapons synthesized because labs will refuse to do so, while it could be much easier to have some novel AI-generated molecule synthesized because the labs don't know what it does.
Seems easily countered by using the same toxicity prediction software when evaluating synthesis requests (but I'm not sure whether this actually matters, or whether skilled chemists can easily synthesize anything themselves anyway).
That's part of it. There's also the risk of clandestine operators discovering easily made chemical weapons that do not require a professional lab to create them in bulk. A nation state, or well funded terrorist group could exploit such a thing without too much effort.
It's important to remember. Chemical warfare can be used for mass destruction, but this approach could be used for other nefarious things at smaller more discrete scales en masse. ie 1000 attacks with different agents in each one... Forensic nightmare.
I like your suggestion to counter these things, but, these are predictive tools. They can and often are or will be wrong. False positives would be a real problem. Again though the people interested in doing this won't be dialing up a chemical supplier to do it for them.
Seems easily countered by using the same toxicity prediction software when evaluating synthesis requests (but I'm not sure whether this actually matters, or whether skilled chemists can easily synthesize anything themselves anyway).