The prompt is kv-cached, it's precomputed. | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		GaggiX on Aug 27, 2024 \| parent \| context \| favorite \| on: Anthropic publishes the 'system prompts' that make... The prompt is kv-cached, it's precomputed.

WithinReason on Aug 27, 2024 [–]

Good point, but it still increases the compute of all subsequent tokens

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact