Carbon Calculations
Methodology for calculating the environmental impact of AI inference operations.
Our comprehensive methodology for calculating the environmental impact of AI inference operations.
Alpha version notice: currently we're building up our calculations primarily based on technical specifications of our infrastructure and assumptions regarding our actual usage. We're looking for cases to compare our approach with. Are you measuring, calculating, or estimating the energy consumption and environmental footprint of your infrastructure? Let's have a chat!
Sustainability metrics overview
Our two-step process for measuring environmental impact.
Step 1: estimating energy
- Estimate GPU power consumption.
- Estimate CPU power consumption.
- Factor in Power Usage Effectiveness (PUE).
Step 2: estimate environmental impact
- Convert energy consumption to CO₂ equivalent.
Practical implementation
- Measure inference time.
- Estimate GPU energy consumption.
- Factor in CPU energy consumption to get "cluster energy consumption".
Energy estimation methodology
Based on measured inference runtime and hardware specifications.
Calculation factors
We multiply measured inference runtime by three key factors:
- GPU power (kW): based on hardware specifications and load assumptions.
- CPU overhead multiplier: includes GPU-supporting CPUs and cluster application CPUs.
- Power Usage Effectiveness: datacenter efficiency factor.
GPU power calculations
NVIDIA H100 PCIe specifications and power assumptions.
Hardware specifications
- GPU: NVIDIA H100 PCIe.
- TDP rating: 350 W.
Power consumption assumptions
TDP represents a maximum threshold rather than actual usage. Our estimates are based on real-world measurements:
| State | Power | Description |
|---|---|---|
| Idle power | ~100 W | Baseline consumption when not processing. |
| Under load | ~300 W | Typical inference workload consumption. |
Note: these estimates are based on expert analysis. The H100 can potentially draw more than its TDP rating under certain conditions.
CPU overhead calculations
Estimating total cluster energy consumption beyond GPU usage.
GPU instance CPUs
H100_1_80G instances are packaged with AMD Zen 4 CPUs. Power
consumption estimates:
| State | Power | Description |
|---|---|---|
| Idle | ~40 W | Baseline CPU consumption. |
| Feeding GPU | ~80 W | During inference operations. |
Open question: vCPU allocation methodology: determining whether to use simple proportion (vCPU / CPU-cores) for shared resources.
Non-inference workload CPUs
POP2_4C_16G instances use AMD EPYC 7543 32-Core Processors (TDP:
225 W):
| State | Power | % of TDP |
|---|---|---|
| Idle | ~33.75 W | 15% |
| Typical load | ~67.5 W | 30% |
Overhead factor calculation
Methodology for scaling from GPU-only to total cluster energy consumption.
Usage assumptions
- Typical load: 4 hours / day.
- Idle time: 20 hours / day.
Calculation method
- Step 1, calculate
E_GPU: energy consumption estimation for GPU-only operations (excluding GPU-supporting CPUs). - Step 2, calculate
E_Cluster: total energy consumption including GPUs and all CPU overhead.
Overhead factor formula
Overhead Factor = E_Cluster / E_GPUThis proportion allows us to scale GPU energy consumption to total cluster consumption.
Implementation
The overhead factor is applied to measured inference times to estimate the total environmental impact of each AI operation, including all supporting infrastructure.