In January 2025, a relatively unknown lab released a set of open-weight reasoning models that performed close to the best closed APIs at a fraction of the training cost—and the whole industry stopped to recalculate. Overnight, "should we use an open-source model?" went from a fringe question asked by infrastructure teams to a boardroom one. The momentum was real, but the conversation around it was mostly noise: benchmark screenshots, cost claims with no context, and a lot of strong opinions about a decision that is, in the end, an engineering and economics problem. As Softechinfra's CTO, I get this question from founders almost weekly, and the honest answer is never "open is better" or "just use the API." It is a decision tree. This guide is that tree: the durable criteria for choosing between open-weight models you host and closed models you rent, the real cost math, and the part most teams skip until it hurts—who is responsible when the model is yours.
What "Open-Source AI" Actually Means
The phrase is doing a lot of work, and most of the confusion starts here. There are three distinct things people mean, and conflating them leads to bad decisions.
Open weights. The trained model parameters are downloadable and you can run them on your own hardware. You usually do not get the training data or the full training code. Most "open" models businesses care about—the Llama family, Mistral's releases, Qwen, Gemma, the new DeepSeek weights—fall here. This is what matters operationally: you can self-host.
Open source (strict). Weights, training code, and data are all released under a recognized open license. This is rarer and mostly relevant to researchers; for a business deploying a model, it changes little day to day.
Open license vs. restricted license. "Open weights" does not automatically mean "free for any use." Several popular releases carry acceptable-use clauses, commercial thresholds, or attribution requirements. Read the license before you build a product on it—this is a legal review item, not a footnote.
The Real Trade-Off: Control vs. Convenience
Strip away the hype and the choice reduces to one axis. Closed APIs sell you convenience: someone else handles serving, scaling, uptime, and the next model upgrade. Open weights sell you control: over where your data goes, what the model can be tuned to do, and what it costs at scale. Everything else is detail hanging off that spine.
| Dimension | Open-Weight (Self-Hosted) | Closed API |
|---|---|---|
| Data privacy | Data never leaves your infrastructure | Sent to a third party; governed by their terms |
| Cost at scale | Fixed infra cost; cheap per call at high volume | Pay-per-token; predictable, expensive at volume |
| Cost at low volume | Idle GPUs still cost money | Near-zero when not in use |
| Fine-tuning control | Full—tune freely on your data | Limited to what the vendor exposes |
| Operational burden | You own serving, scaling, uptime | Vendor handles it |
| Capability ceiling | Strong, trailing the frontier slightly | Typically the highest raw capability |
| Vendor lock-in | None—weights are portable | Real; switching cost is non-trivial |
Notice there is no column that wins every row. A team processing sensitive medical records weights privacy above all and will accept operational burden to keep data in-house. A two-person startup shipping a prototype weights convenience and should not be running GPUs. The right answer is whichever model's strengths line up with the constraints that actually bind your business.
The Self-Host Economics Most Teams Get Wrong
The seductive pitch is "tokens are basically free once you host your own model." It is true at the margin and badly misleading as a total. Self-hosting trades a variable per-token cost for a largely fixed infrastructure cost, and that trade only pays off above a volume threshold.
The honest cost of self-hosting includes far more than the GPU rental line:
- GPU compute, billed whether or not requests are flowing—idle capacity is pure loss
- Engineering time to deploy, optimize, and keep the serving stack patched
- Autoscaling and load balancing for traffic spikes, or over-provisioning to avoid them
- Monitoring, logging, and on-call coverage when inference degrades at 2 a.m.
- Model evaluation and re-validation every time you swap in a newer open release
There is a crossover point. Below it, an API is cheaper because you pay only for what you use and someone else amortizes the hardware. Above it—steady, high request volume—a well-utilized self-hosted model wins on unit economics, sometimes dramatically. The mistake is assuming you are above the line when you are nowhere near it. Most early products run sporadic, bursty traffic, which is the worst possible profile for owned GPUs.
A Decision Framework You Can Run This Week
When a client asks us whether to go open or closed for an AI automation build, we work through five questions in order. Answer them honestly and the choice usually makes itself.
- Is data privacy a hard constraint? If regulation, contracts, or genuine sensitivity mean data cannot leave your infrastructure, self-hosting an open-weight model moves to the top of the list regardless of cost. This single answer overrides most others.
- What is your real inference volume? Estimate steady-state requests per day. Low or bursty points to an API. High and predictable makes self-hosting's economics start to work.
- Do you need deep fine-tuning? If your edge depends on a model tuned heavily on proprietary data or specialized to a narrow task, open weights give you control that closed APIs only partly expose.
- How capable does the model need to be? For the hardest frontier reasoning, the top closed models still lead. For well-scoped, repetitive tasks—classification, extraction, routing, summarization—a good open model is often more than enough.
- Do you have the operational muscle? Be brutally honest. Running production inference is a real engineering commitment. If you don't have the team or the appetite, an API is not a compromise—it's the correct answer.
A pattern worth naming: these are not mutually exclusive. Many mature systems are hybrids—a closed API for the hardest 10% of queries that demand peak capability, a self-hosted open model for the high-volume, well-defined 90% where cost dominates. Routing between them by task complexity is often the most economical architecture of all, and it keeps you from being locked into a single vendor's roadmap.
When an API Still Wins
Because the open-source momentum is loud right now, it's worth stating plainly when renting beats owning. Reach for a closed API when you are early and validating an idea—do not provision GPUs for a product that might pivot next month. Reach for it when your traffic is low or spiky, when you need the absolute frontier of capability, or when your team's time is better spent on product than on inference infrastructure. "We used an API" is not a failure of ambition; for most teams shipping their first AI feature, it is the disciplined choice. We unpack the broader sequencing of these calls in our guide to enterprise AI transformation.
The Part Everyone Skips: You Own the Behavior Now
Here is the responsibility that comes free with the weights, and it is the line item most plans forget. When you self-host an open model, you inherit everything a vendor's safety and reliability team used to handle for you. There is no provider absorbing the liability for a harmful output, no upstream team quietly patching a jailbreak, no SLA to fall back on. The model's behavior in production is yours.
In practice that means an evaluation harness on your own representative tasks rather than trusting public benchmarks, output monitoring and logging so you notice drift and failures, a rollback plan for when a model upgrade quietly regresses on something you cared about, and a clear human-in-the-loop boundary for high-stakes decisions. The discipline of testing a model on your tasks rather than chasing leaderboards is the same one we apply across every model choice; it is the core idea in our guide to evaluating new AI models, and it matters even more when the model is yours to operate.
A grounded example: on ExamReady, the exam-preparation platform we built, the AI that generates and grades practice questions is held to an internal eval suite of curated questions with known-correct answers—because a model that quietly drifts on grading erodes student trust faster than any outage. That harness, not a benchmark screenshot, is what tells us whether a given model is fit to ship. The same principle holds on TalkDrill, our in-house English-speaking practice app, where we evaluate speech and language models on real learner audio before any of them touch production.
Putting It Together
The post-DeepSeek excitement was a useful jolt because it forced teams to look at a real option they had been ignoring. But the durable lesson is not "open won." It is that open-weight and closed-API models are now both mature enough to be genuine choices, and the right one depends entirely on your constraints: privacy, volume, fine-tuning needs, capability bar, and operational capacity. Run the five-question framework, do the volume math before provisioning anything, and budget for the responsibility you take on when the behavior becomes yours. Build the strategy on those criteria and it will outlast every model release that follows—including the ones launching while you read this.
Choosing an AI Model Strategy for Your Product?
We help teams pick between open-weight and closed models, run the cost math, and build the evaluation and serving infrastructure that makes the choice hold up in production.
Talk to Our AI Team →
