Sorry, I realized I didn't quite write what I meant to. I didn't intend to say t... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		Imnimo 89 days ago \| parent \| context \| favorite \| on: Markov chains are the original language models Sorry, I realized I didn't quite write what I meant to. I didn't intend to say that LLMs are non-Markovian from a theoretical standpoint. I meant to say that the language generation task is non-Markovian from a theoretical standpoint, because the next word can depend on arbitrarily distant history.

vrighter 89 days ago [–]

it can depend on whatever the oldest thing in the context window is, but not anything older

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact