Using spare GPU power for one-person LLM inference

The issue is how to make better use of a graphics card when only one person is running local LLM inference. The setup appears to produce about 45 tokens per second for one person, but it can also serve 4 people at the same time at about 40 tokens per second each.

That means a single request may leave much of the card’s total capacity unused, while multiple requests can keep it busier. The practical question is whether one person can turn that spare capacity into more tokens per second.