LLM Inference Energy Cost Calculator

What this LLM inference energy calculator estimates

LLM inference is the part of the lifecycle that keeps generating cost after the model is trained. Each prompt, reply, retrieval hop, tool call, and background API request consumes GPU time, so the recurring bill can grow long after the initial training run is over. This calculator is built to make that operational side visible. It turns a handful of practical inputs into daily estimates for runtime, electricity use, carbon emissions, and direct serving cost. The goal is not to hide the mechanics behind a single headline number, but to let you see how traffic, model size, and hardware efficiency combine into a real deployment expense.

The estimates are intentionally tuned for planning conversations about LLM inference, not for simulating every detail of a production stack. You enter the model size in billions of parameters, the average token load per request, the number of requests served each day, and the approximate throughput and power draw of the GPUs doing the work. The calculator then estimates how much compute those requests imply, how long the hardware has to stay busy, and how that runtime converts into kilowatt-hours, kilograms of CO₂, and dollars. It is a compact way to compare deployment choices, sanity-check a budget, or explain why a larger prompt window can have a bigger cost impact than expected.

How to enter LLM inference inputs well

To get a useful LLM inference estimate, begin with one representative request rather than with the whole system at once. Start with model parameters, entered in billions. A dense 7B model should be entered as 7, a 13B model as 13, and so on. Then decide what your tokens per request figure should include. For many planning exercises, the clearest option is total tokens processed per request, meaning prompt tokens plus generated output tokens together. If you only want to study generation cost, you can use output tokens alone, but then the result should be interpreted as a narrower slice of serving demand.

Next, enter requests per day as the amount of traffic you expect the LLM service to handle in a 24-hour period. That can come from logs, a launch forecast, or a scenario you are using to test capacity. Per-GPU compute should reflect the sustained serving figure you expect in practice. Peak marketing TFLOPS can look impressive on a spec sheet, yet the real serving stack may run well below that once batching, memory pressure, latency targets, and sharding overhead are included. If you have an observed benchmark from a similar deployment, use it. If not, a conservative value usually gives a safer first estimate than an optimistic one.

The last three inputs connect LLM inference runtime to money and emissions. GPU cost per hour can be a cloud on-demand rate, a reserved equivalent, or an amortized on-prem figure that bundles hardware and facility costs into one hourly price. GPU power draw should represent the watts used during inference rather than the absolute maximum from a board specification. If you want to account for cooling or other facility overhead without adding extra fields, you can raise the wattage input slightly. Finally, grid CO₂ intensity converts energy into carbon by expressing how many kilograms of CO₂ are associated with each kilowatt-hour in your region or provider mix.

1. Enter the dense model parameter count in billions.
2. Enter average tokens per request, ideally prompt plus output together.
3. Enter daily request volume.
4. Enter per-GPU TFLOPS and the number of serving GPUs.
5. Enter hourly GPU cost, power draw, and grid carbon intensity.
6. Click Estimate Inference Cost to see daily totals and the detailed breakdown.

After the calculation runs, the Copy Result button copies a short summary line. That makes it easy to drop the headline energy, cost, and emissions figures into a note, a spreadsheet, or an email without retyping them.

LLM inference formula and assumptions

For dense LLM serving, the calculator uses a simple first-pass approximation that is common in workload sizing. The assumption is that each token requires work that scales roughly with the number of model parameters. A widely used planning rule is about 2 × N FLOPs per token, where N is the parameter count. That shortcut is not a full architecture model, and real systems can sit above or below it depending on implementation details, but it is transparent and easy to reason about when you are comparing prompts, models, or deployment plans.

In those equations, N is the parameter count, T is tokens per request, R is requests per day, and G is the number of GPUs. Once compute hours are known, the rest of the arithmetic is straightforward. Carbon equals energy multiplied by grid intensity. Daily cost equals GPU hourly price multiplied by GPU count and runtime. The calculator also derives throughput in tokens per second and requests per second, then compares that capacity against your daily traffic to show utilization.

The reason to keep the model simple is clarity. In early LLM inference planning, you often need to know whether model size or prompt length matters more than electricity price, or whether more efficient hardware will help both cost and emissions enough to justify the switch. A transparent estimate is easier to discuss than a more complicated model whose assumptions are buried in the background. For many decisions, getting the direction and the rough scale right is more useful than pretending to have a perfect simulator.

How to interpret the LLM inference result

The headline result gives three daily numbers: energy, cost, and CO₂. Energy is the number to watch when you are planning power draw, comparing GPU efficiency, or estimating how much extra load a service will place on a data center. Cost translates the same runtime into a budget figure that product and finance teams can act on. CO₂ becomes most useful when you are choosing between regions or providers with different electricity mixes. Because all three outputs are driven by the same runtime estimate, changes that reduce one usually help the others as well.

The table underneath adds context that is especially relevant for LLM inference. FLOPs per request shows the amount of compute implied by one average request. Compute time per day turns that compute into total GPU-hours of active work. Max requests per second is a rough capacity estimate based on the current token assumption. Utilization compares your daily workload with that capacity. If utilization is very high, you may be close to latency risk when prompts get longer or traffic spikes. If utilization is very low, you may be provisioning much more hardware than the average load actually needs.

Two smaller outputs are often useful when you are discussing unit economics for LLM inference: cost per request and cost per 1k tokens. Those values translate infrastructure math into product pricing language. If a team wants to know what happens when a prompt grows from a few hundred tokens to several hundred more, the per-request and per-1k-token figures usually make the tradeoff easier to understand than a daily total alone.

Worked example: serving a 7B LLM with 50,000 daily requests

Suppose you are serving a 7B model, your average request uses 200 tokens, and your service handles 50,000 requests per day. The deployment runs on 8 GPUs with an effective serving throughput of 150 TFLOPS each. The GPUs draw about 300 watts each during inference, the blended price is $2.50 per GPU-hour, and the electricity available to the deployment corresponds to 0.40 kg CO₂ per kWh.

Using the 2N rule, the model needs about 14 billion FLOPs per token. Multiply that by 200 tokens and then by 50,000 requests to estimate the total daily work. Divide by the combined GPU throughput to convert that work into runtime. Once runtime is known, energy becomes power multiplied by time, cost becomes hourly rate multiplied by time, and carbon becomes energy multiplied by grid intensity. The calculator performs those steps instantly, but the logic stays visible so you can see which levers matter most. In this kind of LLM inference estimate, model size, token count, and sustained throughput are usually the strongest drivers of the final answer.

If the result looks higher or lower than expected, the first thing to check is usually the token count. Many teams underestimate total tokens by looking only at output length and forgetting prompt length, system instructions, retrieved context, or hidden tool traffic. The second thing to check is whether the TFLOPS figure reflects actual serving conditions instead of a theoretical peak. Those two adjustments often explain most of the gap between a quick estimate and the bills or telemetry from a real deployment.

Important limitations in LLM inference energy estimates

This calculator is best treated as a planning aid for LLM inference, not as a substitute for measurements from an operating system. Real serving stacks are shaped by much more than raw multiply-add throughput. Memory bandwidth, batching policy, queueing strategy, latency targets, quantization, interconnect limits, KV cache reuse, speculative decoding, sparsity, and prompt length distribution all affect the result. Some of those factors reduce the work needed per token, while others limit the throughput you can actually sustain.

• Dense-model approximation: the 2N rule is most natural for dense transformer workloads and can misrepresent MoE or sparsity-heavy models.
• Peak versus sustained performance: published TFLOPS can be much higher than what a serving stack achieves under production latency constraints.
• No explicit idle billing: the estimate is based on active compute time. If GPUs stay provisioned around the clock, the real bill may be higher.
• Facility overhead is simplified: CPUs, memory, networking, storage, load balancers, and cooling are not broken out separately.
• Traffic variation matters: a daily average can hide burstiness. A system that looks fine on average may still need more headroom at peak hours.

None of those caveats make the calculator less useful. They simply define the scope of the estimate. It is strongest when you want a defensible first pass, when you want to compare LLM inference scenarios on equal footing, or when you need to explain why prompt growth or hardware choice changes the budget and emissions curve.

Reference table: FLOPs per token by model size

For quick LLM inference calibration, the table below uses the same 2N approximation as the calculator. It is a fast way to sanity-check how much work a request implies before you even plug in traffic volume. Even at this coarse level, it is easy to see why model choice has such a large impact on serving economics.

Practical notes for better LLM inference estimates

For realistic planning, it helps to run several LLM inference scenarios instead of relying on a single number. Try a typical day and a busy day. Try a p50 token count and a p95 token count. Try a smaller model and a larger one. Compare a region with a cleaner grid and a region with a more carbon-intensive grid. The real value is not just the final estimate, but the sense of which inputs dominate the outcome. In many deployments, prompt discipline and model choice deliver larger savings than tiny changes in hourly GPU price.

If you want to approximate data center overhead, a simple trick is to inflate the power draw input with a PUE-like adjustment. For example, if you believe total facility energy is about 1.3 times GPU energy, multiply the wattage input by 1.3 before calculating. That keeps the page simple while still bringing the estimate closer to measured energy use. Likewise, if your cloud bill includes idle capacity, compare the calculator's active-runtime estimate with a 24-hour reserved-capacity estimate to bracket the likely real cost of your serving setup.

The most useful habit is consistency. If your team always counts prompt plus output tokens the same way, always uses the same sustained TFLOPS assumption for a given hardware profile, and always documents whether power includes overhead, then repeated runs of the calculator become a practical planning tool rather than just a one-off number generator.