Module 01 · Why Run Models Locally
Installing Ollama and Running Your First Local Model
Open lesson + course map
On this lesson
Course outline
Module 1 · Why Run Models Locally
Module 2 · Hardware Fundamentals
Module 3 · GPU Benchmarking
Module 4 · CUDA and the GPU Software Stack
Module 5 · Local Inference Optimization
Module 6 · Choosing and Running Models
Module 7 · VRAM Optimization
Module 8 · Deploying Local AI Infrastructure
Install Ollama from its official channel, start with a small model, and record enough system information to reproduce the result. Model names, versions, hardware support, and commands can change, so use the current official documentation and library rather than copying an old social post.
// concept
Preflight the Machine
Record operating system/build, CPU, system RAM, GPU model and VRAM, free disk, driver version, and whether virtualization or WSL is involved. Back up important work. Confirm that organizational policy permits local model downloads and that disk encryption, user accounts, and antivirus controls are appropriate.
Download Ollama from the official site for Windows or macOS, or follow the official Linux instructions. Review the publisher and signature/checksum information offered by your platform. Do not paste an installation command from an unverified mirror or run as administrator without understanding why it needs that permission.
After installation, open a new terminal:
ollama --version
ollama listIf the command is missing, restart the terminal and inspect the installed path. If the service is not available, use the official troubleshooting/log instructions instead of repeatedly reinstalling random driver packages.
// concept
Select and Run a Small Model
Visit Ollama’s official model library, read the model page, license/terms, sizes, capabilities, and prompt notes, then choose a variant that fits the machine. The following command is a pattern; replace the identifier with a currently listed model you have reviewed:
ollama run <approved-model:variant>The first run downloads weights, so verify available disk and network. When the prompt opens, use a harmless test such as:
Return exactly this JSON object with no extra keys:
{"status":"ok","sum":7}
Calculate 3 + 4 before responding.Save the model identifier, digest shown by local tooling, Ollama version, prompt, response, time to first token, total duration, and observed CPU/GPU/RAM use. A correct result proves only that this one test ran.
// concept
Check the Local API Carefully
Ollama exposes a local API. Confirm the documented address and keep it bound to the local machine during learning. Test with non-sensitive text:
$body = @{ model='<approved-model:variant>'; prompt='Reply with OK'; stream=$false } | ConvertTo-Json
Invoke-RestMethod -Uri 'http://localhost:11434/api/generate' -Method Post -ContentType 'application/json' -Body $bodyDo not bind the service to every network interface, forward a router port, or assume a home LAN is trusted. Network exposure is a later security task.
// concept
Troubleshoot Methodically
If the model is slow or fails, capture the exact error, available memory before/after, logs, and ollama ps output. Try a smaller approved model or shorter context. Change one variable at a time. A GPU appearing in the computer does not prove the runtime is using it; inspect the processor/offload information and vendor monitoring.
// pakistan_angle
Pakistan Angle
Large weight downloads can be expensive or unreliable on capped/mobile connections. Download one justified variant, preserve its exact identifier, and avoid repeatedly deleting it. Plan updates during reliable connectivity and keep enough disk headroom for temporary files.
Load-shedding can interrupt a download or model run. Use safe power protection suited to the machine, shut down cleanly, and validate files after interruption. Never defeat electrical safety or cooling controls to keep an inference job running.
// hands_on
Hands-On Exercise
Install Ollama from the official source, run one reviewed small model, execute five fixed prompts, and produce a run sheet with system inventory, versions, model identifier, resource observations, outputs, errors, and cleanup command. Use only synthetic data.
// completion_rubric
Completion Rubric
3 grading bands
- Complete
installation source, system inventory, model terms, version, five tests, resource use, and local-only API behavior are documented.
- Needs revision
the model runs but provenance, reproducibility, or hardware-use evidence is missing.
- Not complete
untrusted installers, confidential prompts, or public network exposure are used.
// sources
Sources
4 official sources — check every claim yourself