AI Infrastructure & Local LLMs
0/24 complete

Module 08 · Deploying Local AI Infrastructure

Cost Comparison: Local Infrastructure vs. Cloud API Spend

20 minfocused lesson1copyable prompts3completion checks4source links
Open lesson + course map

On this lesson

Course outline

Compare local and cloud costs with the same measured workload, quality target, reliability level, and time horizon. Cloud pricing can change by model, input/output/cache, region, storage, and service. Local cost changes with utilization, hardware life, energy, cooling, financing, repairs, and staff. There is no universal break-even point.

// concept

Normalize the Workload

Record monthly requests, input/output tokens or task items, peak concurrency, context distribution, uptime, latency, data transfer/storage, and quality threshold. Model retries, invalid outputs, and human review because they consume money even if the provider does not label them “inference.”

Benchmark local candidates on the same evaluation. A cheaper local model that fails the task is not equivalent to a stronger cloud service. If capability differs, compare cost per successful reviewed task rather than raw token.

// concept

Build the Cloud Model

Use official current pricing on the decision date. Include input, cached input, output, embeddings/tools, storage, network, batch discounts only when eligible, taxes/foreign transaction, exchange rate, rate-limit headroom, engineering, observability, and expected retries.

Create low/base/high volume cases. Do not assume a provider price or model remains available for the whole horizon. Record update and migration risk.

// concept

Build the Local Model

// prompt — copy me5 lines
upfront = rig + shipping/tax + power protection + setup
monthly fixed = financing + space + staff + monitoring + internet
monthly variable = measured kW × hours × applicable PKR/kWh + cooling + consumables
monthly risk allowance = expected repairs/downtime/spares (explicit assumption)
effective monthly = fixed + variable + risk + upfront net of residual value / useful-life months

Include idle power and utilization. Local capacity bought for peaks may sit unused. Model replacement hardware lead time and backup path. Do not count the same staff salary twice or hide unpaid founder time.

// concept

Add Reliability and Risk

Compare data handling, compliance review, vendor dependency, local outage, internet outage, power, disaster recovery, model update, security patching, and support. Assign owners and scenario costs without inventing probabilities. A hybrid route may cap cloud spend while preserving an approved fallback.

// worked_example

Worked Example

A team estimates local break-even from GPU price divided by cloud token price. The improved model adds PSU/UPS, electricity, cooling, 20 engineering hours, idle capacity, and replacement risk. It also accounts for a cloud batch price and actual output tokens. Break-even moves materially.

The final decision is a three-month cloud pilot because demand is uncertain, with a local proof running in parallel. This is not a statement that cloud is always cheaper; it reflects reversibility and current evidence.

// failure_cases

Failure Cases

7 cases to diagnose

  • Comparing a local small model to a different-quality cloud model.

  • Omitting output tokens, retries, taxes, exchange, or tools.

  • Omitting idle power, UPS/cooling, setup, and staff locally.

  • Using one electricity bill as a national tariff.

  • Assuming 100% utilization or permanent prices.

  • Calling hardware payback guaranteed revenue.

  • Ignoring backup and downtime.

// pakistan_angle

Pakistan Angle

Date every PKR quote, electricity assumption, and exchange rate. Use the applicable NEPRA/distribution bill or qualified advice rather than a universal tariff. Separate GST/taxes and payment-provider/foreign-card charges as applicable.

Load-shedding, internet reliability, heat, warranty, and import lead time can alter local risk. Cloud dependence also needs reliable international connectivity and foreign-currency payment. Test the actual failure modes and maintain an affordable degraded path.

// hands_on

Hands-On Exercise

Build a 24-month local/cloud/hybrid spreadsheet for one measured workload. Include low/base/high demand, successful-task quality, current official pricing links, dated PKR conversion, complete local ownership, reliability scenarios, sensitivity table, and a reversible three-month recommendation.

// completion_rubric

Completion Rubric

3 grading bands

  • Complete

    equivalent quality/workload, full costs, dated sources, scenarios, risk, residual value, and reversal support the recommendation.

  • Needs revision

    arithmetic is correct but quality equivalence, staff, downtime, or price-change sensitivity is missing.

  • Not complete

    a GPU price/token-rate division is presented as guaranteed break-even.

// sources

Sources