How LLMs work

A Large Language Model does exactly one thing: given the text so far, it predicts a probability for every possible next token. Everything else — answering questions, writing SQL, summarising a supplier contract — emerges from running that single prediction thousands of times, feeding each chosen token back in as input.

Tokens, not words

Text is first split into tokens: word pieces roughly three to four characters long in English. "Ahmedabad" may become two or three tokens; Devanagari or Tamil text usually costs several times more tokens than the same sentence in English, which matters for both cost and context limits. Estimate with our LLM Token Counter before you size a prompt.

Inside the model

Each token becomes a vector (an embedding). Those vectors pass through a stack of identical transformer blocks, each containing two parts:

• Self-attention — every token looks at the other tokens and decides which ones matter for it. This is how the model links "it" back to "the invoice" forty words earlier.
• A feed-forward network — transforms each position independently, storing most of the model's learned facts.

After the final block, a last layer produces one score (logit) per token in the vocabulary. Softmax turns those scores into probabilities, and a sampler picks one.

The knobs you control

Temperature is easiest to feel in plain NumPy:

import numpy as np

logits = np.array([3.0, 2.0, 1.0, 0.5])       # scores for four candidate tokens

def softmax(x, temperature=1.0):
    z = x / temperature
    e = np.exp(z - z.max())
    return e / e.sum()

print(softmax(logits, 0.2).round(3))   # [0.993 0.007 0.    0.   ]  almost deterministic
print(softmax(logits, 1.0).round(3))   # [0.616 0.227 0.083 0.05 ]  more varied

At temperature 0 the model picks the most likely token every time — same prompt, same answer. That is what you want for pulling a GSTIN out of an invoice; it is not what you want for five subject-line variations.

How the model got this way

1. Pre-training — predict the next token over a vast text corpus. This is where grammar, facts and reasoning patterns are absorbed.
2. Supervised fine-tuning — train on curated instruction-and-answer pairs so it follows requests instead of continuing them.
3. Preference tuning — humans (or a model trained on human judgements) rank candidate answers, and the model is nudged toward the preferred style and away from unsafe output.

Why hallucinations happen

Nothing in that process rewards saying "I do not know". The model optimises for a plausible continuation, so when the facts are missing it produces fluent, confidently wrong text. Retrieval (feeding it the real document), asking for citations, and verifying with code or a second call are the practical defences — not a sterner prompt.

Key takeaways

• Next-token prediction plus attention plus scale is the whole mechanism.
• Model names and versions change every few months; build against a capability and keep the model name in config.
• Temperature and max_tokens belong to the task, not to the app; set them per call.
• Fluency is free, truth is not. Ground the model in real data when accuracy matters.