nitpick - EOF is not a signal, it just flushes the current line content without needing a newline in cooked mode and doing this on an empty line results in a 0 byte read(), which most apps interpret as EOF
and the terminal emulator is not itself sending the signals, its always just sending raw characters to the pty device - it is the kernel's pty layer sending a signal if the right flags are on and doing line buffering, local echo, etc. and signals go to the foreground process group (which is set by the shell making a tcsetpgrp syscall)
unfortunately disabling temperature / switching to greedy sampling doesn't necessarily make most LLM inference engines _fully_ deterministic as parallelism and batching can result in floating point error accumulating differently from run to run - it's possible to make them deterministic but does come with a perf hit
some providers _do_ let you set the temperature, including to "zero", but most will not take the perf hit to offer true determinism
possible - but KV caches are generally _much_ bigger than the source text and can be reproduced from the source text so it wouldn't make a lot of sense to throw it out
pcie devices can also do direct transfers to each other - if you have one of these and a gpu its relatively quick to move data between them without bouncing through main ram
Erlang spreads processes across OS threads, like most other green-threading impls, and its not always great at it. (I don't know what you mean by "across OS processes." Processes on different Erlang VMs can communicate but the scheduler isn't going to move processes between them)
Calling into native code isn't an easy problem in Erlang either. NIF calls will block a scheduler thread, but the scheduler knows nothing about how long a NIF call is expected to take and will happily queue up processes to be run on a thread that is blocked inside a NIF call.
"Regular" erlang I/O is done by queueing up requests to be fulfilled by .. you guessed it, a pool of threads that spend most of their time sleeping in blocking i/o calls.
and the terminal emulator is not itself sending the signals, its always just sending raw characters to the pty device - it is the kernel's pty layer sending a signal if the right flags are on and doing line buffering, local echo, etc. and signals go to the foreground process group (which is set by the shell making a tcsetpgrp syscall)
reply