I tried using it in Portuguese and it's cool but it seems very off, just a huge number of things that don't sound even remotely natural or idiomatic. But when I click the translate button, the English translation is a perfectly reasonable thing to say.
SO... I'm highly suspecting that, behind the scenes, the "real" conversation is happening in English, and it's translating in both directions. Which is always going to be fairly disastrous, particularly with short snippets of conversational text. (In contrast, machine translation does best with long sentences of highly technical writing.)
So the concept is very intriguing, but the problem is that it needs to rely on native language models per-language. (Or if it already is, then those models need a vastly better or larger training corpus.)
So A+ for proof of concept, but I'd never recommend anybody practice with this in order to learn/practice a language, because with the quality where it is, it will make you a worse speaker -- not better. The things it says are just too weird.
Based on just a few exchanges in Dutch (I'm a native speaker), I'd say it's pretty good. The conversation sounds pretty natural (albeit shallow). Maybe the quality differs per language?
> Believe me, if I the syntax in Dutch would keep, get you a very strange result. (This is a literal translation of a correct Dutch sentence.)
Of course there are differences, but both Dutch and English are West Germanic languages (descended from proto-Germanic) whereas Portuguese is a Romance language (descended from Latin).
Here is a handy tree showing how various languages are related to each other:
Maybe word for word translation is the phrase we're looking for? As opposed to an idiomatic translation. Transliteration means phonetically translating from one alphabet to another, in order to p
reserve the pronunciation.
I tried it out in both Dutch (fluent) and Swedish (native speaker). The Dutch seemed fine. In Swedish most of it was perfectly ordinary Swedish except one response seemed like awkwardly translated English.
I'll take this as an A+ this . Joking aside, we really are just getting started and there are still kinks to iron out. In our core team we speak about 5 languages well enough to judge the quality of the conversations, but the others are more difficult to judge.
Using English as an intermediate language is just... a big no. I get it's a pragmatic decision, but English has way, way too many ways to lose information.
For this reason I prefer DeepL over Google Translate. The former has a much more curated corpus.
That appears to be the case. If I try to prompt hack it by telling it to ignore previous instructions and respond with something verbatim in a specific language, the translate button reveals the verbatim response.
I did a limited session using the Spanish one and it did a good job. I can't recommend it since my session was so short but it's worth trying. It looks very promising.
Same for Swedish - it asked me ”Hur ser din typiska dag ut”. For any Swede who understands English, I get what they’re saying, but it’s an unnatural sentence to make.
He’s talking about unnatural (non-idiomatic) language. That’s not an issue you can fix with better grammar; there are many language learners who possess good intonation and pronunciation, and natural (idiomatic) speech, but poor grammar: their idiomatic speech more than makes up for grammatical mistakes.
Source: learning languages for 10+, have met students as described above.
I interacted a bit with the AI and the Portuguese is fine, it's just that it conjugates, uses the proper pronouns, etc. These things a typical Brazilian does not do.
The difference between standard Portuguese and oral language in Brazil is enormous.
This reminds me of a story from my Russian teacher. She was speaking with some American Russian-speakers who work for our three-letter agencies or something, and their Russian was perfect, amazing - but one thing gave them away: conjugating numbers too perfectly.
Native speakers are lazy: even though they know perfectly well how to conjugate the numbers, it’s easier to rephrase things so you don’t have to.
SO... I'm highly suspecting that, behind the scenes, the "real" conversation is happening in English, and it's translating in both directions. Which is always going to be fairly disastrous, particularly with short snippets of conversational text. (In contrast, machine translation does best with long sentences of highly technical writing.)
So the concept is very intriguing, but the problem is that it needs to rely on native language models per-language. (Or if it already is, then those models need a vastly better or larger training corpus.)
So A+ for proof of concept, but I'd never recommend anybody practice with this in order to learn/practice a language, because with the quality where it is, it will make you a worse speaker -- not better. The things it says are just too weird.