Independent research on LLM inference and evaluation reliability.
Tests whether strict grammar-constrained decoding recovers the tool-calling accuracy small models lose when quantized to 4 bits.
Headline finding: Llama-3B Q4: 0% to 88.3% on parallel calls after changing the output format (non-randomized sensitivity analysis).
Tests when reloading a conversation’s saved KV cache is faster than recomputing it, using a break-even bandwidth rule across T4, L4, A10/A10G and H100.
Headline finding: Frozen predictions correctly classified 574/636 resolved decisions on L4 after about a minute of calibration.
Measures how much GPU nondeterminism changes LLM benchmark answers and rankings.
Headline finding: Default greedy decoding changed 2.0–4.6% of answers; a near-tied ranking reversed between GPUs.