A private LLM for business is a large language model that runs where you control the data: on your own servers, on GPUs rented inside your cloud account, or through a model API reached over a private network link with contractual no-training and retention terms. In October 2026, open-weight models such as gpt-oss-120b, Qwen3.8-27B and Mistral Small 4 are good enough for most internal assistants and document work, and they fit on one or two GPUs. Self-hosting only beats API pricing on cost at high, steady volume; for most companies the stronger reasons are data control, residency and predictable spend.
Teams in Germany, Switzerland, the UK, the US and the UAE ask the same three questions: which deployment model, which hardware, and is it cheaper than an API? The honest answers depend on token volume, licence terms and how much operations work you can carry.
This guide compares the three deployment models, sizes the hardware with current GPU prices, lists open models that fit with their licences checked, works through a cost-benefit table against API usage with the maths shown, and sets out the security and audit controls a private deployment needs. It is about running and adapting existing models. If you want to pre-train your own, see this guide on how to build an LLM from scratch.
Key Takeaways
- Three ways to go private. On-premise gives full control and the highest fixed cost; private cloud GPUs give control with rented hardware; a VPC API gives frontier models with private networking but someone else's servers.
- One GPU now runs a capable model. gpt-oss-120b fits on a single 80 GB GPU, and a 27B to 31B dense model fits on one 96 GB workstation card.
- Read the licence before the benchmark. Llama 4's multimodal rights are not granted to companies based in the EU, and Mistral Medium 3.5 bars companies with over $20 million monthly revenue.
- Break-even is higher than vendors suggest. In the worked example, self-hosting costs about $3,700 a month and only beats a mid-priced API above roughly 285,000 requests a month.
- Security is mostly plumbing. Network isolation, document-level permissions in retrieval, verified model weights and logged access matter more than the model choice.
What Is a Private LLM for Business?
A private LLM is a deployment pattern, not a type of model. The same open-weight model can run on a laptop, a server in your data centre or a rented GPU in Frankfurt. What makes it private is who can see the prompts, the documents and the outputs.
Most business deployments combine three parts:
- A model, usually an open-weight model or a frontier model reached through a private endpoint.
- Retrieval over company documents, so answers come from your data rather than the model's memory.
- A control layer for identity, permissions, logging, rate limits and policy.
Retrieval does most of the heavy lifting. Fine-tuning helps with tone, format and narrow tasks, but it rarely beats well-built retrieval for company knowledge. The trade-offs are covered in depth on the page for RAG development services.
On-Premise vs Private Cloud vs VPC API
| Factor | On-premise LLM | Private cloud (your GPUs in your cloud account) | VPC API (provider model, private link) |
|---|---|---|---|
| Where the model runs | Your servers, your building or colocation | Rented GPUs in a region you choose | The provider's infrastructure |
| Who sees prompts | Only your systems | Your account, the cloud provider under its terms | The model provider under its contract |
| Model choice | Open-weight models only | Open-weight models only | Frontier closed models, some open ones |
| Upfront cost | High (hardware) | Low | None |
| Monthly cost | Power, space, staff, depreciation | GPU hours plus staff | Per token |
| Scaling | Buy more hardware | Add instances | Instant |
| Air-gapped option | Yes | No | No |
| Time to first use | Weeks to months, including procurement | Days | Hours |
| Best for | Classified, regulated or air-gapped data; very high steady volume | Data residency with flexible capacity | Teams that need frontier quality and can accept contractual controls |
A VPC API is the option most companies overlook. AWS, for example, lets you reach Amazon Bedrock through interface VPC endpoints powered by PrivateLink, so traffic never crosses the public internet. Azure and Google Cloud offer equivalent private endpoints. The data still leaves your servers, so the provider's retention and training terms decide whether it is private enough.
Which one fits?
- Choose on-premise if data may not leave your premises by law or contract, or if you run large steady batch workloads all day.
- Choose private cloud if you need open models in a specific region (for example Frankfurt or Zurich) but cannot justify buying hardware.
- Choose a VPC API if answer quality on hard tasks matters most and a data processing agreement with no-training terms satisfies your risk team.
Many companies end up hybrid: an open model on private infrastructure for sensitive documents, and a VPC API for general tasks.
Open Models That Fit, With Licences Checked
The table lists open-weight models that suit business use in October 2026, with licence terms read from each model card or licence page. Parameter counts come from the published weights.
| Model | Size | Licence | Watch out for |
|---|---|---|---|
| gpt-oss-120b | 117B total, 5.1B active | Apache 2.0 | Text only; fits one 80 GB GPU in its native MXFP4 format |
| gpt-oss-20b | About 21B | Apache 2.0 | Runs within 16 GB of memory; good for laptops and edge servers |
| Qwen3.8-27B | 27B dense | Apache 2.0 | 262,144-token native context; larger Qwen3.8 variants use other licences |
| Gemma 4 31B | About 31B | Apache 2.0 | Earlier Gemma versions used custom Gemma terms, so check the version |
| Mistral Small 4 | 119B total, 6.5B active | Apache 2.0 | Needs about 120 GB at 8-bit, so two 80 to 96 GB GPUs |
| Mistral Medium 3.5 | 128B | Modified MIT | No rights if your company's monthly revenue exceeds $20 million |
| Llama 4 Scout | 109B total, 17B active | Llama 4 Community License | Multimodal rights not granted to EU-based companies; 700 million user threshold; "Built with Llama" notice |
| DeepSeek V4.1 Flash | Hundreds of billions of parameters (MoE) | MIT | Needs a full 8-GPU data centre node |
| GLM-5.2 | About 753B | MIT | Same node-scale hardware as DeepSeek |
The Llama 4 trap matters for European buyers. Meta's Llama 4 acceptable use policy states that, for multimodal Llama 4 models, the licence rights are not granted to an individual domiciled in, or a company with its principal place of business in, the European Union (end users of a product are not affected). Llama 4 Scout and Maverick are multimodal, so a German company should not deploy them itself. Switzerland and the UK are outside the EU, but group structures can change the answer. The Llama 4 licence also requires a separate licence above 700 million monthly active users and a visible "Built with Llama" notice.
Apache 2.0 is the cleanest choice for most companies, because it allows commercial use and modification and includes a patent grant.
Hardware Sizing With Current GPU Prices
The sizing rule
GPU memory has to hold the model weights plus the KV cache, which grows with the number of concurrent users and the context length.
- Weights: parameters x bytes per parameter. BF16 uses 2 bytes, FP8 uses 1 byte, 4-bit uses about 0.5 bytes.
- KV cache and overhead: budget at least 20 to 50 percent on top of the weights for a few dozen concurrent users with long retrieval contexts.
Worked example: Qwen3.8-27B
The published weights hold about 27.78 billion parameters.
- BF16: 27.78 x 2 = 55.6 GB
- FP8: 27.78 x 1 = 27.8 GB
- 4-bit: 27.78 x 0.5 = 13.9 GB
At BF16, 55.6 GB plus headroom fits one 96 GB card or one 80 GB data centre GPU. At FP8 it fits a 48 GB card with room for many users.
What the hardware costs, as of October 2026
| Hardware | Memory | Price | Source |
|---|---|---|---|
| NVIDIA RTX PRO 6000 Blackwell Workstation Edition | 96 GB GDDR7, 600 W max power | $16,000 per card | NVIDIA Marketplace, NVIDIA product page |
| Cloud H100 SXM, single GPU | 80 GB | $4.29 per GPU hour | Lambda on-demand pricing |
| Cloud H100 SXM, 8-GPU node | 8 x 80 GB | $3.99 per GPU hour | Lambda |
| Cloud B200, 8-GPU node | 8 x B200 | $6.69 per GPU hour | Lambda |
| Cloud A100 80 GB, 8-GPU node | 8 x 80 GB | $2.79 per GPU hour | Lambda |
Workstation card prices have risen sharply through 2026, so get a current quote before you budget. Data centre servers with eight H100, H200 or B200 GPUs are sold through partners on quotation.
Matching models to hardware
| Workload | Model class | Minimum hardware |
|---|---|---|
| Team of 10 to 50, internal Q&A | gpt-oss-20b, Gemma 4 12B | One 24 to 48 GB GPU |
| Company of 100 to 500, assistant plus document search | gpt-oss-120b, Qwen3.8-27B, Gemma 4 31B | One or two 80 to 96 GB GPUs |
| Heavy agent or batch workloads | Mistral Small 4, several replicas of the above | Two to eight GPUs |
| Frontier-class open model | DeepSeek V4.1 Flash, GLM-5.2 | One 8-GPU H200 or B200 node |
Cost-Benefit: Private LLM vs API, With the Maths
The scenario
A 200-person company runs an internal assistant with retrieval. Each employee sends 40 requests per working day, 21 days a month. Each request carries about 4,000 input tokens (question plus retrieved passages) and returns about 500 output tokens.
- Requests: 200 x 40 x 21 = 168,000 a month
- Input: 168,000 x 4,000 = 672 million tokens
- Output: 168,000 x 500 = 84 million tokens
Option A: public or VPC API
Using OpenAI's published October 2026 rates:
- gpt-6.1-sol at $2 input and $10 output per million: 672 x $2 + 84 x $10 = $1,344 + $840 = $2,184 a month
- gpt-5.4-mini at $0.75 and $4.50: 672 x $0.75 + 84 x $4.50 = $504 + $378 = $882 a month
Prompt caching and batch processing can cut these further. Regional data residency endpoints add a 10 percent uplift on newer models.
Option B: on-premise
These are estimates; replace the assumptions with your own quotes.
- Two RTX PRO 6000 cards: 2 x $16,000 = $32,000, plus about $12,000 for the server, storage and networking = $44,000
- Depreciation over 36 months: $44,000 / 36 = $1,222 a month
- Power: about 1.2 kW average x 730 hours = 876 kWh, at an assumed €0.25 per kWh = about $240 a month
- Operations: a quarter of an MLOps engineer at an assumed $9,000 a month loaded cost = $2,250 a month
- Total: about $3,712 a month
Option C: private cloud GPU
- One H100 running 24/7: $4.29 x 730 = $3,132 a month, plus a smaller operations share
- Business hours only (12 hours x 22 days = 264 hours): $4.29 x 264 = $1,133 a month, if you accept start-up delays
Where the lines cross
| Monthly requests | API (gpt-6.1-sol) | API (gpt-5.4-mini) | On-premise (2 GPUs) |
|---|---|---|---|
| 168,000 | $2,184 | $882 | about $3,712 |
| 285,000 | about $3,712 | about $1,500 | about $3,712 |
| 2,000,000 (batch documents) | about $26,000 | about $10,500 | about $4,900 with four GPUs |
The blended API cost per request is (4,000 x $2 + 500 x $10) / 1,000,000 = $0.013 on gpt-6.1-sol and $0.00525 on gpt-5.4-mini. Dividing the $3,712 fixed cost by those figures gives break-even points of about 285,000 requests a month against gpt-6.1-sol and 707,000 against gpt-5.4-mini.
The conclusion is less exciting than most vendor claims. For a typical internal assistant, an API is cheaper. Self-hosting wins on cost when you process millions of documents, run agents around the clock or already have GPU staff. It wins on control whenever data cannot leave your environment, whatever the volume. Quality also differs: open models are strong on summarisation, extraction and retrieval answers, while frontier APIs still lead on the hardest reasoning tasks.
Security and Audit Set-Up
A private deployment removes one risk (a third party seeing your data) and adds others (you now run the stack). These controls belong in every on-premise LLM architecture:
| Control | What to implement |
|---|---|
| Network isolation | Inference servers in a private subnet with no outbound internet; access only through an internal gateway |
| Identity | SSO with MFA for every user; service accounts for every application |
| Document-level permissions | Retrieval filters by the user's existing access rights, so the model never sees files the user could not open |
| Model provenance | Download weights from the publisher's official repository, verify checksums, prefer safetensors over formats that can execute code |
| Patching | Track security releases for the inference server, drivers and container images |
| Logging and audit | Record who asked what, which documents were retrieved, and which tools ran; protect logs from editing |
| Retention | Set and enforce retention for prompts, outputs and logs |
| Prompt injection defences | Treat retrieved documents and emails as untrusted input; limit what tools the model can call |
| Evaluation | Regular red-team tests and accuracy checks on your own question sets |
Prompt injection deserves special care once a model can call tools or read email. The controls in this guide to AI agent security apply equally to self-hosted models.
Country Notes: Germany, Switzerland, UK, US and UAE
- Germany: logging employee prompts can make the assistant a technical device able to monitor behaviour or performance, which triggers works council co-determination under section 87(1) no. 6 of the Works Constitution Act (BetrVG). Agree logging scope and access with the works council before launch.
- Switzerland: the revised Federal Act on Data Protection (FADP) applies to personal data in prompts and documents; on-premise or Zurich-region hosting simplifies cross-border transfer questions.
- United Kingdom: the ICO's guidance on AI and data protection expects a data protection impact assessment for high-risk processing, which most HR or customer data use cases will be.
- United States: there is no single federal privacy law, so sector rules (HIPAA, GLBA, state privacy laws) and customer contracts usually drive the private deployment decision.
- UAE: federal personal data protection law and free zone regimes such as DIFC and ADGM apply their own rules on transfers, which makes in-country or on-premise hosting attractive for regulated firms.
Governance sits above all of this. A private model still needs an owner, a risk tier and a register entry, as set out in this AI governance framework covering policy, risk and oversight.
A Private LLM Decision Checklist
Answer these in order. The first "yes" usually decides the deployment model.
- Must data stay on premises by law, contract or classification? Choose on-premise.
- Must data stay in a specific country, and do you need open models? Choose private cloud in that region.
- Do you need frontier reasoning quality, and does a no-training DPA satisfy your risk team? Choose a VPC API.
- Is volume above about 300,000 requests a month, or are you running batch jobs on millions of documents? Model self-hosting costs seriously.
- Do you have, or can you hire, someone to run GPUs, patch servers and monitor quality? If not, prefer private cloud or a managed option.
- Does the licence of your chosen model allow your company, your country and your use? Check before any proof of concept.
How to Roll Out a Private LLM in 90 Days
- Weeks 1 to 2: pick two or three high-value use cases, gather 100 real questions with good answers, and choose the deployment model.
- Weeks 3 to 6: stand up the inference server and retrieval pipeline, connect SSO and document permissions, and test two candidate models on your question set.
- Weeks 7 to 10: pilot with 20 to 50 users, log quality issues, tune retrieval and prompts, and run a security review.
- Weeks 11 to 13: roll out to the wider company with usage dashboards, a feedback button and a monthly evaluation cycle.
How Pixel Web Solutions Can Help
Pixel Web Solutions designs and builds private LLM deployments on-premise, in private cloud regions and behind VPC endpoints, including retrieval, permissions, audit logging and evaluation. Its large language model development company services cover model selection with licence checks, hardware sizing, fine-tuning where it helps and integration with existing systems. The team operates from Ras Al Khaimah, UAE and Madurai, India, and works with clients in Europe, the UK, the US and the Gulf. Custom engineering starts from $40 an hour. To size a private LLM for your workload, contact Pixel Web Solutions.
Frequently Asked Questions
What is the best private LLM for a small business?
For most small businesses, gpt-oss-20b or a 12B to 31B model such as Gemma 4 runs well on a single GPU and carries the permissive Apache 2.0 licence. If you need stronger reasoning, gpt-oss-120b fits on one 80 GB GPU. Many small businesses are better served by a VPC API with no-training terms.
How much does it cost to run an LLM on premise?
In the worked example, two 96 GB workstation GPUs, a server, power and a quarter of an engineer cost about $3,700 a month over three years. Smaller teams can start with a single GPU workstation; frontier-scale open models need an 8-GPU server sold on quotation.
Is a self-hosted LLM cheaper than ChatGPT or an API?
Usually not at low volume. In the example, a mid-priced API costs $882 to $2,184 a month for 168,000 requests, against about $3,712 for self-hosting. Self-hosting becomes cheaper at several hundred thousand requests a month or for large batch jobs.
What hardware do I need for an on-premise LLM deployment?
Multiply the parameter count by bytes per parameter (2 for BF16, 1 for FP8, 0.5 for 4-bit) to get the weight memory, then add 20 to 50 percent for the KV cache. A 27B model at BF16 needs about 56 GB plus headroom, so one 80 or 96 GB GPU.
Can a private LLM be GDPR compliant?
Yes. Keeping inference on your own infrastructure or in an EU region removes third-country transfer issues for the model itself, but you still need a lawful basis, data minimisation, retention limits and access controls for prompts, documents and logs.
Should I fine-tune a private LLM or use RAG?
Start with retrieval. It keeps answers tied to current documents and respects permissions. Fine-tune when you need a consistent format, a specialised vocabulary or a narrow task that retrieval cannot teach.
Can I use Llama 4 in Germany?
Not as a deploying company for Llama 4's multimodal models: Meta's acceptable use policy withholds those licence rights from companies with their principal place of business in the EU. Apache 2.0 models such as gpt-oss, Qwen3.8-27B, Gemma 4 or Mistral Small 4 avoid the issue.
A private LLM for business makes sense when control, residency or very high volume matter more than the convenience of a public API. Size the hardware from the model's weights and your users, check the licence before the benchmark, run the cost maths with your own token counts, and build the permissions and audit trail in from the start.
