Let’s start where a fair comparison has to start: the DIY route is real. Onyx, PrivateGPT and their siblings are serious open-source projects. Open-weight models are genuinely good now. A capable engineer can stand up a private document-QA stack on a weekend and demo it on Monday.
That demo is where most build-vs-buy decisions are made — and it is the most misleading data point in the whole journey.
The demo is 10% of the project
A stack that answers questions about ten PDFs in a demo and a system your estimating office relies on every day are different things wearing the same interface. Between them sits the unglamorous list:
- Ingestion that survives your real documents. Scanned PDFs, spreadsheets with merged cells, exports from an estimating suite, fifteen years of inconsistent naming. Every format that parses wrong is a class of questions that silently returns nothing.
- Retrieval tuning. Out of the box, semantic search retrieves plausible passages. Whether “steel pipe 57×3.5” reliably finds its eleven other spellings in your catalogues is a matter of chunking, embedding choice and evaluation — per corpus, not per install.
- Evaluation. Without a reference sample and a measured accuracy number, you don’t know whether the system works; you know whether it feels impressive. Building that harness is real engineering work.
- Operations. Model updates every few months, GPU drivers, backups, access control, monitoring, and someone on call when the answer engine stops answering at 9 a.m. on tender day.
- The person who leaves. The stack lives in one engineer’s head. Their resignation letter is your migration project.
None of this is a criticism of the open-source projects. They provide the engine. You are building the car, the garage and the maintenance schedule.
Where the line actually sits
Our honest rule of thumb — and we sell the alternative, so discount accordingly — is that building starts to make sense at around three dedicated ML engineers. At that scale you can afford ingestion work, an evaluation harness, on-call coverage and bus-factor redundancy, and you get something a vendor can’t give you: a system shaped exactly to your workflow.
Below that line, the arithmetic turns against you quietly. One engineer “doing it on the side” means the stack competes with the work your company actually sells. The costs don’t appear on an invoice — they appear as the feature that took a quarter, the accuracy nobody measured, the upgrade nobody dared to run.
And the alternative costs are not hypothetical: cloud AI seats run $30–90 per user per month, forever, and enterprise document-AI suites start at $100k+ a year and are scoped for Fortune 500 procurement. The DIY stack is usually an attempt to escape exactly those two price shapes. The question is whether you escape into engineering payroll instead.
The appliance answer
Perimeter exists for the companies below the three-engineer line. It runs the same class of open-weight models a DIY stack would — the model is a slot, not an identity — but it arrives as an appliance: a box in your server room, set up on your corpus, with zero external calls.
The parts of DIY that are actually valuable, you keep:
- Ownership. The hardware is bought in the EU in your name. The corpus and the index live on the box, inside your walls. We keep no copies.
- Open models. Open-weight, with public provenance, inspectable. Updates are included in support — when a stronger model ships, we test it on your reference sample and swap it in.
- Verifiability. Pull the network cable. It keeps working.
The parts of DIY that are a second job, you hand over: ingestion for your formats, retrieval tuning, the evaluation harness, the load test, and the acceptance gate — a threshold agreed in writing on your own sample, before any licence is invoiced. Live in 30 days, without hiring anyone.
The honest summary
If you have three engineers to dedicate and a workflow no product fits — build. You will own every screw, and that has real value.
If what you actually need is answers with sources from documents that can’t leave your building — the engineering project is the expensive way to buy an appliance.
The one-page version of this comparison, including the rows where the cloud and DIY win, is in the comparison matrix.