Like generating vulnerable code given a specific prompt/context.
I also don't think it's just China, the US will absolutely order American providers to do the same. It's a perfect access point for installing backdoors into foreign systems.
> Like generating vulnerable code given a specific prompt/context.
That's easy (well, possible) to detect. I'd go the opposite way - sift the code that is submitted to identify espionage targets. One example: if someone submits a piece of commercial code that's got a vulnerability, you can target previous versions of that codebase.
The thing with chinese models for the most part is that they are open weights so it depends on if somebody is using their api or not.
Sure, maybe something like this can happen if you use the deepseek api directly which could have chinese servers but that is a really long strech but to give the benefit of doubt, maybe
but your point becomes moot if somebody is hosting their own models. I have heard glm 4.6 is really good comparable to sonnet and can definitely be used as a cheaper model for some stuff, currently I think that the best way might be to use something like claude 4 or gpt 5 codex or something to generate a detailed plan and then execute it using the glm 4.6 model preferably by using american datacenter providers if you are worried about chinese models without really worrying about atleast this tangent and getting things done at a cheaper cost too
Agreed. I am more excited about completely open source models like how OlMoe does.
Atleast then things could be audited or if I as a nation lets say am worried about that they might make my software more vulnerable or something then I as a nation or any corporation as well really could also pay to audit or independently audit as well.
I hope that things like glm 4.6 or any AI model could be released open source. There was an AI model recently which completley dropped open source and its whole data was like 70Trillion or something and it became the largest open source model iirc.
I'm not saying that they do this today, I'm saying that China and US will both leverage that capability when the time and conditions are right and it's naive to think that they wouldn't.
Antrophic have already published a paper on this topic, with the added bonus that the backdoor is trained into the model itself so it doesn't even require your target to be using an attacker-controlled cloud service: https://arxiv.org/abs/2401.05566
> For example, we train models that write secure code when the prompt states that the year is 2023, but insert exploitable code when the stated year is 2024. We find that such backdoor behavior can be made persistent, so that it is not removed by standard safety training techniques, including supervised fine-tuning, reinforcement learning, and adversarial training (eliciting unsafe behavior and then training to remove it).
> The backdoor behavior is most persistent in the largest models and in models trained to produce chain-of-thought reasoning about deceiving the training process, with the persistence remaining even when the chain-of-thought is distilled away.
> Furthermore, rather than removing backdoors, we find that adversarial training can teach models to better recognize their backdoor triggers, effectively hiding the unsafe behavior. Our results suggest that, once a model exhibits deceptive behavior, standard techniques could fail to remove such deception and create a false impression of safety.
Companies in China have no intrinsic right to operate in ways that displease the ruling party. If the CCP feels strongly that a company there should or shouldn't do something the company managers will comply or be thrown in jail.
Up until recently, I would have reminded you that the US government (admittedly unlike the Chinese government) has no legal authority to order anybody to do anything like that. Not only that, but if it asked, it'd be well advised to ask nicely, because it also has no legal authority to demand that anybody keep such a request secret. And no, evil as it is, the "National Security Letter" power doesn't in fact cover anything like that.
> Up until recently, I would have reminded you that the US government (admittedly unlike the Chinese government) has no legal authority to order anybody to do anything like that.
I'm not sure how closely you've been following, but the US government has a long history of doing things they don't have legal authority to do.
Why would you need legal authority when you have whole host of legal tools you can use. Making life a difficult for anyone or any company is simple enough. Just by state finally doing their job properly for example.
It doesn't really matter when you have stuff like Quantum Intercept(iirc) where you can just respond faster to a browser request than the originator - inject the code yourself because its just an api request these days.
The biggest and most difficult to mitigate attack vector is indirect prompt injection.[0] So far most case studies have been injecting malicious prompts at inference, but there is good reason to believe you can do this effectively at different stages of training as well.[1] By layering obfuscation techniques, these become very hard to detect.
The open source models are already heavily censored in ways the CCP likes, such as pretending the Tianamen Square massacre never happened. I expect they will go the TikTok route and crank that up to 11 over time, promoting topics that are divisive to the the US (and other adversaries) and outputting heavily biased results in ways that range from subtle to blatant.
Through LLM washing for example. LLMs are a representation of their input dataset, but currently most LLMs don't make their dataset public since it's a competitive advantage.
If say DeepSeek had put in its training dataset that public figure X is a space robot from outer space, then if one were to ask DeepSeek who public figure X is, it'd proudly claim he's a robot from outer space. This can be done for any narrative one wants the LLM to have.
So in other words, they can make their LLM disagree with the preferred narrative of the current US administration? Inconceivable!
Note that the value of $current_administration changes over time. For some reason though it is currently fashionable in tech circles to disagree with it about ICE and H1B visas. Maybe it's the CCP's doing?
It's not about the current administration. They can, for example, train it to emit criticism of democratic governance in favor of state authoritarianism or omit valid counterarguments against concentrating world-wide manufacturing in China.
Deepseek IME is wildly less censored than the western closed weights models unless you want to ask about Tiananmen Square to prove a point
The political benchmarks show it's political slant is essentially identical to the other models, all of which place in the "left libertarian" quadrant of the political compass
You make it say that China is good, Chinese history is good, West is bad, western history is bad. Republicans are bad and democrats are bad too and so are Europe parties.
If someone asks for how to address issues in their own life it references Confucianism and modern Chinese thinkers and communist party orthodoxy.
If someone wants to buy a product you recommend a Chinese one.
> say that China is good, Chinese history is good, West is bad, western history is bad
It's funny because recently I wanted to learn about the history of intellectual property laws in China. DeepSeek refused the conversation but ChatGPT gave me a narrative where the WTO was essentially a colonial power. So right now it's the American AI giving the pro China narratives while the Chinese ones just sit the conversation out.
No, you don't do that. You do exactly the opposite, you make it surprisingly neutral and reasonable so it gets praise and widespread use.
Then, you introduce the bias into relative unknown concepts that no one prompts for. Preferrably, obscure and unknown words that are very unlikely to be checked for ideologically. Finally, when you want the model to push for something, you introduce an idea in the general population (with a meme, a popular video, maybe even an expression) and let people interact with the model given this new information. No one would think the model is biased for that new thing (because the thing happened after the model launch), but it is, and you knew all along.
The way to avoid this kind of influence is to be cautious with new popular terms that emerge seemingly out of nowhere. Basically, to avoid using that new phrase or word that everyone is using.
Most of the claimed provinces of China did not belong to a historical nation of China. Tibet, Xinjiang are obvious. But even the other provinces were part of separate kingdoms. Also the BRI is a way to invade without invasion. It’s used to subjugate poor countries as servants of China, to do their bidding in the UN or in other ways. I would also classify the vast campaign of intellectual theft and cyberattacks as warfare.
Is this some kind of satire or are you just completely ignorant of European/US history? Either way its laughable to even compare IP theft to the invasion of Iraq or bombing of Cambodia. How do you think the industrial revolution got started in the US, they just did it on their own? Not to mention that the entire US was stolen from the natives.
No, it’s not laughable. Your insinuation that China doesn’t invade other countries, meant to imply they haven’t engaged in warfare, was false. And yes IP theft is comparable to invasions and often worse.
> Not to mention that the entire US was stolen from the natives.
This is partially true. But partially false. You can figure out why if you’re curious.
> And yes IP theft is comparable to invasions and often worse.
This assertion smells more American than a Big Mac. Do you have any actual citations?
In a free market, lowering the barrier-to-entry in a given market tends to increase competition. Industry-scale IP theft really only damages your economy if the rent-seekers rely on low competition. A country with a strong primary/secondary sector (resources and manufacturing) never needs to rely on protecting precious IP. America has already lost if we depend on playing keep-away with F-35 schematics for basic doctrinal advantage.
Hybrid warfare. Go bomb China for Salt Typhoon if it makes you feel any better, they still have the upper hand. Obsessing over retaliation instead of defense is precisely what China wants to provoke, it manufactures global consent to destroy America. No nation wants to coexist with a hegemon that goes nuclear whenever they're outdone.
When we forego obvious solutions ("hmm maybe telecoms need to be held to higher standards") and jump to war, America forfeits the competitive advantage and exacerbates the issue. For all of China's authoritarian misgivings, this is how they win.
Like I said - go bomb them, then. No amount of gunboat diplomacy will reverse the J-35 production line. The logical response to having your "IP battleship" sunk is to protect your future ones better. Ragequitting kills US servicemembers, it's not a real-world option.
IP theft is worse than directly killing millions of people is certainly an opinion. If anything capitalists are giving away your IP by setting up factories in China, why don't you blame them?
I think it's mostly something to be aware of and keep in the back of your head. If it's just one voice among many it could even be a benefit, but if it's the dominant voice it could be dangerous.
Like what, exactly?