Linear & logistic regression intuition

Linear models are the "hello world" of machine learning, but they are also workhorses in banks, insurers and retail chains because they are fast, stable and easy to explain to a manager. If you understand them well, every other model becomes easier to reason about.

Linear regression: fitting the best straight line

Linear regression predicts a number as a weighted sum of features:

price = w1 * area + w2 * floor + w3 * age + b

Training means finding the weights w and intercept b that minimise the mean squared error between predictions and actual values. Each weight answers a simple question: if this feature goes up by one unit and everything else stays the same, how much does the prediction change?

import pandas as pd
from sklearn.linear_model import LinearRegression

flats = pd.DataFrame({
    "area_sqft": [650, 900, 1100, 1250, 1500, 1800],
    "age_years": [12, 8, 5, 10, 3, 2],
    "price_lakh": [48, 70, 92, 95, 128, 155],   # Pune flats, INR lakh
})
X, y = flats[["area_sqft", "age_years"]], flats["price_lakh"]

model = LinearRegression().fit(X, y)
print(dict(zip(X.columns, model.coef_.round(3))), round(model.intercept_, 2))
print(model.predict(pd.DataFrame({"area_sqft": [1000], "age_years": [6]})))

A coefficient of about 0.08 on area_sqft would mean every extra square foot adds roughly 0.08 lakh (INR 8,000).

Logistic regression: probabilities, not numbers

For yes/no problems (will this customer default on an EMI?) we still compute a weighted sum z, but pass it through the sigmoid function 1 / (1 + e^-z), which squashes any number into the range 0 to 1. The output is a probability; a threshold (0.5 by default) turns it into a class. It is trained by minimising log loss, which punishes confident wrong answers heavily.

from sklearn.linear_model import LogisticRegression
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler

clf = make_pipeline(StandardScaler(), LogisticRegression(C=1.0, max_iter=1000))
clf.fit(X_train, y_train)
proba = clf.predict_proba(X_test)[:, 1]          # probability of class 1
risky = proba > 0.3                              # business-chosen threshold

Coefficients in logistic regression act on the log-odds: exp(coef) gives the odds ratio. A value of 1.5 means one unit more of that (scaled) feature multiplies the odds of default by 1.5.

Regularisation in one table

Common mistakes

• Forgetting to scale features before regularised models, so the penalty unfairly hits large-unit columns.
• Reading coefficients as "importance" when features are on different scales or strongly correlated.
• Using linear regression for a 0/1 target; predictions can go below 0 or above 1.
• Assuming a straight line fits curved relationships. Add polynomial or log features, or switch models.
• Always using a 0.5 threshold even when a missed default costs far more than a false alarm.

Key takeaways

Linear models are your baseline: train one first, and only accept a complex model if it clearly beats it on validation data. They are also the easiest to explain to auditors and business stakeholders.