I feel like there is no portable advice for performance. A torch model exported as onnx is a different model.

That onnx model run using onnxruntime with cuda ep is a different model than the one run with TRT ep.

And even among the same runtime, depending on the target hardware and the memory available during tuning, the model behaves differently. It is a humongous mess

That's interesting as I was considering GGUF --> ONNX conversions (via Olive), but if this creates unknown distortions in the effectiveness and stability, it might be a dead-end idea.

Just to clarify: I mean VRAM, RAM and runtime performance, not the numerical outputs (even though those also vary to some degree, haha)

don't know about this guy, but qwen3.6:27b with the UD 4bit quant and little-coder/pi has been amazing. the first local LLM experience that can do actual meaningful work

What is UD?

Unsloth Dynamic, just some branding from Unsloth for their quants (other people use similar techniques)

Maybe most interesting about the piece is, that we'll likely see more large scale interviews like this (even if this one is a bit bland)

very nice. has a claude touch to it

this is.. a nothing burger? they don't exclude working for autonomous weapons, nor do they exclude mass surveillance. so what gives?

the headlines these days

for me gpt-oss:20b was that. glm 4.7 flash was not better, but much slower on a 16GB card

wow, everything is exactly unfolding as some AI doomers have projected

Anki is and was truly a blessing. Not sure I would have gotten through my studies without it. Thank you dae!