Pre-training
The first stage of training a foundation model on very large, broad datasets to learn general knowledge and language.
Pre-training data is measured in trillions of tokens. Most licensed private data is used in later stages, where smaller, specialist sets have more effect.
Related terms
Tell us what you need
Book a free 30-minute consultation. Tell us what data you want to buy or license, and we'll tell you honestly whether we can help and what it involves.
- We reply to every enquiry within two working days.
- No upfront fees for sellers
- We never hold your raw data