The short answer
Buy off the shelf when a catalog dataset already matches your task, language and label schema closely enough to pass your evaluation, because it is faster and usually cheaper. Commission when it does not, when you need ownership or exclusivity, or when the behavior you want has never been recorded. Test a sample of the catalog set before deciding.
On this page
Off-the-shelf datasets sit in vendor catalogs ready to license: speech in dozens of languages, labeled street scenes, question and answer pairs, expert-written reasoning examples. Custom datasets are made for you, to your guidelines. The first is faster; the second fits. The choice is less about price than about how close the catalog comes to your task, and you can measure that before spending much.
Who sells off-the-shelf data
Several of the AI training data companies that do custom work also keep catalogs. TELUS Digital sells ready-made sets beside its custom services, and Surge AI offers prebuilt expert datasets next to its RLHF and evaluation work. Defined.ai runs a marketplace selling speech, image, video, music and text datasets from its data suppliers. General data marketplaces such as AWS Data Exchange and Snowflake Marketplace list datasets whose prices are set by the providers.
Catalog prices are mostly not published. Expect to ask for a sample and a quote.
When off the shelf is the right call
The task is common
Speech recognition in a major language, object detection for common classes, general instruction following: these have been built many times, and a catalog set may already cover them well.
Your schema is flexible
If you can adapt your labels to the catalog’s, or map one onto the other without losing meaning, you save the cost of guidelines, pilots and review.
You need it soon
A licensed catalog set can be delivered as soon as the agreement is signed. A commissioned set needs recruiting, guideline writing and a pilot before volume work begins.
When to commission instead
The behavior has never been recorded
A model that must handle your company’s own document types, your product’s failure modes or an agent workflow in your software needs examples that do not exist anywhere else. RL environments are the extreme case: Epoch AI heard that a graded task usually sells for $200 to $2,000, because each one is built from nothing.
You need ownership or exclusivity
Catalog sets are licensed to many buyers. If the data is meant to be an advantage, or your customers require that you own your training data, commissioning with an ownership clause is the cleaner route.
Quality requirements are strict
A catalog set’s labels were made to someone else’s guidelines and accuracy target. If yours differ, relabeling a catalog set can cost nearly as much as commissioning a new one.
How the costs compare
| Off the shelf | Commissioned | |
|---|---|---|
| Up-front work | Sample review and license negotiation | Guidelines, recruiting, paid pilot |
| Pricing basis | License fee for the set, often non-exclusive | Per item, per hour or fixed price per deliverable |
| Published examples | Rare; catalogs quote on request | Label Your Data: bounding boxes from $0.02 per object, about $6 per annotator hour |
| Time to first data | After the license is signed | After the pilot is accepted |
| Rights you get | A license for a term | Ownership, if the contract says so |
| Biggest hidden cost | Relabeling to fit your schema | Rework when annotators drift |
For a first figure on the commissioned side, the cost estimator multiplies your volume by published per-unit rates. Treat its output as a floor.
Questions to put to a catalog vendor
- When was the data collected, and has it been refreshed since?
- Which annotation guidelines were used, and can we read them?
- How many other buyers have licensed the same set?
- Can we license a subset, or only the whole catalog item?
- What happens to models we train if the license is not renewed?
The answers tell you how far the set sits from your task before any sample arrives, and they double as the first page of your provenance file.
A test that settles most cases
- Ask the catalog vendor for a sample of a few hundred items, under a short evaluation agreement.
- Train or fine-tune on the sample, or simply score it against your guidelines.
- Measure the share of items you would accept as they are, the share you would relabel, and the share you would discard.
- Price the catalog option as the license plus relabeling plus discards. Price the custom option from a pilot quote.
If more than about a fifth of the sample needs relabeling, the catalog set’s advantage usually disappears, because relabeling runs through the same review loop as new work. That threshold is a rule of thumb, not a published figure; set your own from the pilot.
The hybrid most teams end up with
The common answer is both: a catalog set for breadth, licensed non-exclusively, and a smaller commissioned set for the cases that matter most to your users. The commissioned part is usually where the model’s distinctive behavior comes from, and the part worth owning.
Buy the parts of the problem that everyone shares, and commission the parts that only you have.