Task guide · open weights
Best local open-weight AI: decide what local actually means
Open weights, self-hosting, private inference, and a local workstation are different claims. This guide separates them before you buy hardware.
What actually matters
- Check the licence, weights availability, quantisation, serving stack, and hardware footprint together.
- A model being open-weight does not mean it fits a laptop or permits every commercial use.
- Compare total cost of ownership: GPUs, power, engineering, monitoring, and upgrade cycles.
- Use a hosted API first when the model is datacentre-scale and your reason for going local is untested.
- Coding agents need long context and tool-call reliability — current Ollama guidance recommends at least 64K for agent/coding tasks when hardware supports it; verify the context actually allocated and test the chosen model's tool calls.
- Match the API “door”: Claude Code wants Anthropic Messages–compatible endpoints; OpenCode-class tools want OpenAI /v1.
- Start local agents scaffold-first: map one repository, make one documentation or test edit, and inspect tool calls before attempting a multi-file build.
- Treat VRAM as a floor for the chosen quantisation and context, not a marketing maximum; use the /gpus hub to compare usable memory before buying.
The shortlist
- DeepSeek V4.1-Flash for a low-cost open option with a hosted API and MIT-licensed weights (it replaced the retired V4-Flash on 2026-09-10).
- Kimi K3 when a 1-million-token context and open weights fit the workload, and its custom Kimi K3 License is acceptable; the model card describes a 2.8T-parameter checkpoint.
- Small, specialised releases such as LFM2.5 when on-device latency is the actual goal.
- Aider or Continue configured with a local model for offline assistive edits; treat full agent loops on local models as scaffold-first experiments, not drop-in GPT replacements.
Evidence-backed entries
A sensible test workflow
- 01Write down the privacy and deployment requirement in one sentence.
- 02Estimate memory, throughput, quantisation quality, and serving complexity before downloading weights.
- 03Run a representative prompt set locally and against the hosted alternative.
- 04Before blaming the model, verify context length, endpoint compatibility, and whether telemetry retries are hammering a local server.
- 05Repeat the scaffold-first task at the target context length and concurrency before trusting a local agent with broad edits.
Common mistakes
- Calling an API-only preview open source because weights were promised.
- Ignoring the licence or model-specific commercial conditions.
- Buying hardware before measuring workload throughput and concurrency.
- Assuming a runtime's default or configured context is what the harness actually receives; inspect the allocated context and truncation behavior before judging a local model.
- Pointing Claude Code at an OpenAI /v1 proxy (or OpenCode at an Anthropic-only door) and chasing auth ghosts.
Popular-AI shortlist snapshot: September 26, 2026. This guide is a starting framework, not a permanent ranking. Model prices, access, policies, and capabilities move quickly. Check the provider before committing spend or sending sensitive data.