An open-weight model provides downloadable weights under stated terms; that does not automatically mean its training data, source code, or license satisfies an open-source definition. The landscape changes quickly. Select from dated official model cards and task evaluations, not a permanent “best model” list.
Classify Candidates by Need
Build a candidate register across:
- modality: text, code, image/audio input, embeddings, reranking, or safety;
- size/architecture: dense or mixture-of-experts and active/total parameters where documented;
- task tuning: base/pretrained, instruction, code, domain, or safety variant;
- language and context claims;
- weight precision/formats and supported runtimes;
- license/terms, acceptable-use restrictions, and attribution;
- publisher, official repository, revision, release/update date, and checksum/digest;
- model-card evaluations, limitations, and known risks.
Families from organizations such as Google, Meta, Mistral, Qwen, and others may offer different sizes and terms. Community fine-tunes and quantizations can be useful but add a provenance chain. Treat any list in this July 2026 lesson as examples, not an up-to-date leaderboard; revisit publisher pages when choosing.
Use a Selection Funnel
- Eligibility: license, commercial use, distribution, geography, privacy, and organizational policy.
- Compatibility: runtime, hardware backend, VRAM/RAM, context, and deployment environment.
- Capability: fixed task set including critical languages/formats and failure cases.
- Operations: latency, concurrency, update path, model availability, monitoring, and support.
- Risk: harmful output, prompt injection, data leakage, supply-chain trust, and rollback.
- Cost: full local/cloud ownership under measured demand.
Eliminate ineligible candidates before spending time on speed benchmarks.
Read Model Cards Skeptically
Check intended use, out-of-scope use, languages, training/evaluation information, safety notes, license link, base model, and exact revision. Benchmark scores may use different prompts, harness versions, sampling, hardware, or contamination controls. Reproduce relevant tests and add your application-specific set.
Official status matters, but official models can still be wrong or unsuitable. Model cards communicate known information; they are not a warranty.
Worked Example
A bilingual customer-support assistant needs English and Urdu classification, JSON output, and one-GPU deployment. The team starts with eight candidates, removes those whose terms or hardware do not fit, and tests three on 200 synthetic/authorized examples. It scores per-language accuracy, invalid JSON, critical misroutes, TTFT, memory, and concurrency.
The selected smaller model beats a larger candidate on this narrow task. The decision register names the exact revision and review date. A quarterly trigger reopens selection because runtimes and models change.
Failure Cases
- Calling downloadable weights unrestricted open source.
- Selecting by parameter count or one public leaderboard.
- Ignoring base versus instruction variants and chat templates.
- Running arbitrary repository code with trust flags.
- Missing license changes or downstream quantization terms.
- Testing only English for a bilingual product.
- Automatically upgrading model aliases in production.
🇵🇰 Pakistan Angle
Evaluate Urdu script, Roman Urdu, code-switching, Pakistani names/locations, PKR formats, and local document patterns only when the product needs them. Use fluent reviewers and never infer suitability from a publisher’s broad “multilingual” claim.
Hardware availability and download size matter, but do not trade away license compliance or safety. Cache approved revisions according to terms and keep a fallback for interrupted downloads and model withdrawal.
Hands-On Exercise
Create a dated register of ten current candidates from official publisher cards. Cite every field. Apply eligibility and compatibility gates, then evaluate the remaining two or three on at least 50 task examples. Produce a revision-pinned decision and next-review trigger.
Completion Rubric
- Complete: sources, revisions, terms, compatibility, task quality, safety, and operating evidence drive a dated choice.
- Needs revision: candidates are tested but licensing, provenance, or local-language slices are incomplete.
- Not complete: a “best models” list, popularity, or one leaderboard substitutes for selection.
Sources
- Hugging Face model-card documentation
- Hugging Face repository licenses
- Google Gemma model overview
- Ollama model library
Key takeaway: Model choice is a revision-pinned, license-first workload evaluation that expires as the landscape changes.