Computer vision & NLP in practice

Vision and language are the two areas where AI has changed everyday products the most — from UPI apps scanning cheques to keyboards suggesting Marathi words. Both fields now use the same underlying idea: convert messy input into vectors, then let a trained network map those vectors to the answer you need.

Computer vision tasks

An image is a tensor of shape (channels, height, width). Almost every vision model expects a fixed size and a normalisation that matches its training data:

import torch
from torchvision import transforms
from PIL import Image

preprocess = transforms.Compose([
    transforms.Resize(256),
    transforms.CenterCrop(224),
    transforms.ToTensor(),
    transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),
])

img = Image.open("shelf.jpg").convert("RGB")
batch = preprocess(img).unsqueeze(0)      # shape (1, 3, 224, 224)
print(batch.shape, batch.dtype)

Convolutional networks (ResNet, EfficientNet) and vision transformers both work well; in practice you rarely train from scratch. Transfer learning — reusing a pretrained backbone and retraining only the last layers — reaches useful accuracy with a few hundred labelled photos.

NLP tasks

The pipeline is: normalise → tokenise → embed → model → decode. Tokenisation is where Indian languages get interesting. A subword tokeniser trained mostly on English splits Devanagari or Tamil text into many more tokens than the equivalent English sentence, which raises cost and eats into the context window. Code-mixed "Hinglish" written in Roman script ("bhai order kab aayega") also defeats naive language detection, so test on real messages, not textbook sentences.

Practical notes for Indian data

• Collect images in the conditions you will deploy in: low light, dusty lenses, glare from a shop shutter.
• Invoices and forms vary wildly by vendor; an OCR step plus a layout-aware extractor beats a single regex.
• For speech and text, expect multiple scripts, spelling variants and mixed languages in the same record.
• Always keep the raw file plus the extracted fields, so you can re-run extraction when the model improves.

Common mistakes

• Resizing images at inference with a different normalisation than training, then blaming the model.
• Measuring OCR with "did it read the invoice" instead of per-field accuracy on the fields you actually use.
• Ignoring class imbalance: 2% defective cartons means an "always OK" model scores 98%.
• Treating an English-only tokeniser or stopword list as good enough for Indic text.