There are more aspects to the randomness such as race conditions and intentionally nondeterministic tiebreaking when tokens have the same probability, apparently.

Yeah, in addition to what the commenter above said about floating points and GPU calculations, LLMs are never fully deterministic.

So now you finally admit that LLMs are not truly deterministic and only near-deterministic.

I’ve told you that from the beginning, but you were too smug, to first admit that major LLM provider systems are not deterministic, and then too smug to look up what near-deterministic systems are and do some research, and barking up the wrong tree.