What is a large language model (LLM)?

In practice

Models differ in cost, speed, context length and how well they follow instructions, and the gap between the best and the merely adequate is usually smaller than the gap between a good and a bad prompt, or between good and bad retrieval. Treat the model as a replaceable part: anything in your architecture that assumes one specific model will be expensive to change, and you will want to change it.

Where it fits in a build

Language models fit tasks where the input is unstructured language and the output is a judgement, a summary, a classification or a structured extraction. They do not fit arithmetic, lookups of current facts, or any decision that must be identical every time it is made.

Common mistakes

• Asking a model to do arithmetic or look up a live figure instead of calling the system that holds it.
• Building the prompt into the code in fifty places, so improving it means a release.
• Choosing a model before defining how the output will be evaluated, which leaves nobody able to say whether a change made things better.

Related terms

• Context window
• Fine-tuning
• Prompt engineering

Working on something that involves this?

ASTACKRA designs and builds AI systems, automation and custom software for businesses that need technology shaped around their own workflow. If this term turned up in a proposal and you want a straight answer about whether it applies to your situation, ask us — no obligation, and we will tell you if the answer is no.