Open-source AI is often called free.

“Open-source AI is free” usually mixes up three different things: the license, the model weights, and the cost of running a useful service.

Even when weights are downloadable, the real cost can include:

• GPU or accelerator purchase and depreciation;
• electricity, cooling, storage, and networking;
• inference software, monitoring, updates, and security;
• fine-tuning, evaluation, and engineering time;
• downtime, backups, and the opportunity cost of maintaining the stack.

A recent lifecycle-cost framework, LCOAI, explicitly argues that token price alone misses capital expenditure, inference, storage, monitoring, retraining, DevOps, and the productive outputs delivered over the system’s lifetime.

For a hobbyist with hardware already paid for, a local model may have very low marginal cost. For a company, the right comparison is total cost per accepted task, not cost per token: include failed generations, retries, human review, and the engineer who keeps the service running. APIs turn much of the cost into a variable expense; self-hosting turns it into capital and operational work.

I would also check the license carefully. “Open weights,” “source available,” and “open source” are not automatically synonymous, especially for commercial deployment. The cheapest model is not necessarily the cheapest system.