AI Infrastructure & Local LLMs
0/24 complete

Module 01 · Why Run Models Locally

Installing Ollama and Running Your First Local Model

20 minfocused lesson1copyable prompts3completion checks4source links
Open lesson + course map

On this lesson

Course outline

Install Ollama from its official channel, start with a small model, and record enough system information to reproduce the result. Model names, versions, hardware support, and commands can change, so use the current official documentation and library rather than copying an old social post.

// concept

Preflight the Machine

Record operating system/build, CPU, system RAM, GPU model and VRAM, free disk, driver version, and whether virtualization or WSL is involved. Back up important work. Confirm that organizational policy permits local model downloads and that disk encryption, user accounts, and antivirus controls are appropriate.

Download Ollama from the official site for Windows or macOS, or follow the official Linux instructions. Review the publisher and signature/checksum information offered by your platform. Do not paste an installation command from an unverified mirror or run as administrator without understanding why it needs that permission.

After installation, open a new terminal:

// powershell2 lines
ollama --version
ollama list

If the command is missing, restart the terminal and inspect the installed path. If the service is not available, use the official troubleshooting/log instructions instead of repeatedly reinstalling random driver packages.

// concept

Select and Run a Small Model

Visit Ollama’s official model library, read the model page, license/terms, sizes, capabilities, and prompt notes, then choose a variant that fits the machine. The following command is a pattern; replace the identifier with a currently listed model you have reviewed:

// powershell1 line
ollama run <approved-model:variant>

The first run downloads weights, so verify available disk and network. When the prompt opens, use a harmless test such as:

// prompt — copy me3 lines
Return exactly this JSON object with no extra keys:
{"status":"ok","sum":7}
Calculate 3 + 4 before responding.

Save the model identifier, digest shown by local tooling, Ollama version, prompt, response, time to first token, total duration, and observed CPU/GPU/RAM use. A correct result proves only that this one test ran.

// concept

Check the Local API Carefully

Ollama exposes a local API. Confirm the documented address and keep it bound to the local machine during learning. Test with non-sensitive text:

// powershell2 lines
$body = @{ model='<approved-model:variant>'; prompt='Reply with OK'; stream=$false } | ConvertTo-Json
Invoke-RestMethod -Uri 'http://localhost:11434/api/generate' -Method Post -ContentType 'application/json' -Body $body

Do not bind the service to every network interface, forward a router port, or assume a home LAN is trusted. Network exposure is a later security task.

// concept

Troubleshoot Methodically

If the model is slow or fails, capture the exact error, available memory before/after, logs, and ollama ps output. Try a smaller approved model or shorter context. Change one variable at a time. A GPU appearing in the computer does not prove the runtime is using it; inspect the processor/offload information and vendor monitoring.

// pakistan_angle

Pakistan Angle

Large weight downloads can be expensive or unreliable on capped/mobile connections. Download one justified variant, preserve its exact identifier, and avoid repeatedly deleting it. Plan updates during reliable connectivity and keep enough disk headroom for temporary files.

Load-shedding can interrupt a download or model run. Use safe power protection suited to the machine, shut down cleanly, and validate files after interruption. Never defeat electrical safety or cooling controls to keep an inference job running.

// hands_on

Hands-On Exercise

Install Ollama from the official source, run one reviewed small model, execute five fixed prompts, and produce a run sheet with system inventory, versions, model identifier, resource observations, outputs, errors, and cleanup command. Use only synthetic data.

// completion_rubric

Completion Rubric

3 grading bands

  • Complete

    installation source, system inventory, model terms, version, five tests, resource use, and local-only API behavior are documented.

  • Needs revision

    the model runs but provenance, reproducibility, or hardware-use evidence is missing.

  • Not complete

    untrusted installers, confidential prompts, or public network exposure are used.

// sources

Sources