Research
Measurements I was willing to publish
Cost attribution for shared AI infrastructure, written up so the numbers can be checked. Code, raw data and the hardware runbook ship with every paper.
Who Pays for the KV Cache? Attributing Shared AI Inference Spend Across Kubernetes and LLM Provider Bills
When several teams share one inference server, the bill does not divide itself: somebody picks a metering rule. This paper measures how much that choice moves, and where cost attribution breaks at the seams between Kubernetes allocations, gateway logs and provider billing APIs. Five case studies, including a validation run with vLLM on an NVIDIA H100.
12–14 pts
disagreement between two meters
On an H100 running vLLM, a token meter assigned a retrieval-heavy tenant 12 to 14 percentage points more of the same bill than an equal time-share meter, at every load tested.
97–99%
GPU utilization, while throughput rose 7×
Utilization is a presence metric. It barely moved while real output went from 773 to 5,389 tokens per second.
66%
of a multi-pod GPU bill left unowned
In a constructed month of allocations: owner labels set on the LeaderWorkerSet leader template only. The obvious fallback key then routed 61% of the money to a Helm chart name.
6,241
requests served, zero errors
Four load levels, two minutes each, on one rented H100 for 29 minutes and about $2.15.
Read it
- • Abstract and full text on arXiv · PDF · doi:10.48550/arXiv.2609.24991
- • Code, raw data and the GPU runbook: github.com/timurista/unalloc, archived at Zenodo
- •
pip install unallocinstalls the tool the paper describes
The same work, in plainer language
- • Designing showback for shared LLM inference
- • How the tool was checked before anyone quoted it
- • The case study
What this is, and what it is not
This is a preprint. arXiv moderates submissions; it does not peer review them. Treat it as public, citable work that has not been through review.
No allocation in the paper is a ground truth: the meters compared are defensible rules, and what is measured is how far they disagree, not which share is correct. The GPU result is one run per load level on one model and one GPU. Dollar figures in the attribution scenarios come from constructed months, not from real invoices.
Reviewing or citing this
I would rather hear where it is wrong than where it is interesting. If you run inference or split shared infrastructure cost, the methods section is the part worth attacking.