Also tried it on a small project, it did ok finding issues but completely failed doing rather basic edits, like it lost closing brackets or used wrong syntax and couldn't recover. The CLI was easy to setup and use though.
Did you try it via OpenRouter? If so, what provider? I've noticed some providers seems to not exactly be upfront about what quantization they're using, you can see that the responses from some providers who supposedly run the exact same model and weights give vastly different responses.
Back when Devstral 1 released, this was made very noticeable to me because the ones who used the smaller quantizations were unable to actually properly format the code, just as you noticed, that's why this sounded so similar to what I've seen before.
In my experience, the messed up closing brackets are a surprisingly common issue for LLMs. Both Sonnet 4.5 and Gemini 3 also do this regularly. Seems like something that should be relatively easy to fix, though.
I use both Sonnet 4.5 and Opus 4.5 to edit lisp (emacs lisp to be be precise) and run into this issue extremely infrequently. Not sure if they have some special handling for this but seems to work ok. I have this problem with Gemini, and less frequently, with Qwen.
