How AI systems learn
Every learning system, from a spam filter to a large language model, follows the same four-stage loop. Learn it once and everything afterwards — machine learning, deep learning, LLMs — is a variation on the theme.
The four stages
- Data — examples of the task, ideally collected the way they will actually arrive in production.
- Model — a function with adjustable parameters (weights). It starts out useless.
- Training — measure the error with a loss function, then nudge the parameters to reduce it, repeatedly.
- Inference — freeze the parameters and run the model on inputs it has never seen.
Training is where the learning happens; inference is where the value is. Their costs differ wildly: training may take hours on a GPU, while inference has to answer inside your app in milliseconds.
Four styles of learning
| Style | What the data looks like | Example |
|---|---|---|
| Supervised | Inputs and correct answers | Predict tomorrow's milk demand from last year's sales |
| Unsupervised | Inputs only | Group 50,000 customers into buying segments |
| Reinforcement | Actions and rewards over time | A warehouse robot learning an efficient picking route |
| Self-supervised | Raw data whose own structure supplies the label | Predict the next token of a sentence — how LLMs are trained |
A complete training run
A textile wholesaler wants to predict how many days an order will take to reach the buyer.
from sklearn.linear_model import LinearRegression
from sklearn.model_selection import train_test_split
from sklearn.metrics import mean_absolute_error
# features: [distance_km, items, festival_week]
X = [[12, 4, 0], [180, 9, 0], [45, 2, 1], [320, 15, 0], [8, 1, 0],
[210, 7, 1], [95, 5, 0], [15, 3, 1], [260, 11, 0], [60, 6, 1]]
y = [1, 4, 3, 6, 1, 6, 3, 2, 5, 4] # days actually taken
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)
model = LinearRegression().fit(X_train, y_train)
print("MAE on unseen orders:", mean_absolute_error(y_test, model.predict(X_test)))
print("Predicted days:", model.predict([[150, 8, 1]]))
Ten rows only show the shape of the code; a real model would use thousands.
Generalisation: the only score that counts
A model that memorises the training rows and fails on new ones is overfitting; one too simple to capture the pattern is underfitting. That is why the split above exists — and why you never tune against the test set. Keep a third validation slice for choosing settings, and touch the test set once, at the end.
Data beats cleverness
"Garbage in, garbage out" is not a slogan, it is the usual reason projects fail. Watch for:
- Leakage — a feature that secretly contains the answer, such as
delivered_onwhen predicting delivery days. - Drift — the world moves on; October festival demand does not describe February.
- Imbalance — with 1% fraud, a model that always says "genuine" is 99% accurate and worthless.
- Label noise — two clerks who disagree on what counts as a "damaged" parcel put a ceiling on accuracy.
Common mistakes
- Reporting accuracy on the training data.
- Cleaning or scaling the whole dataset before splitting, so test information leaks into training.
- Reaching for a bigger model when the real problem is 200 mislabelled rows.
- Never retraining: a model is a perishable good, not a one-time build.
Practice
Collect or invent 40 rows of delivery data for your own city and train the model above. Report MAE on the test
split, then add a deliberately leaking column such as days_taken_plus_noise and watch the error collapse — that
is exactly what leakage looks like when it happens by accident.