MediumPython

Top-k frequent words

Return the k most frequent words in a text (case-insensitive).

Word counting is a small problem with real uses. I've used exactly this kind of function to find the most common complaints in customer feedback forms, and to spot which product names people actually type into a search box.

Example:

text = "The cat and the hat. The cat sat."
top_k(text, 2)   # ['the', 'cat']

Things to decide before coding:

"The" and "the" should count as one word, so lowercase everything.
Full stops and commas shouldn't stick to words. "sat." should be "sat".
If two words have the same count, which comes first? This version keeps whichever appeared first in the text, which is what Counter.most_common does.

Starter code

def top_k(text, k):
    pass

💡 Hint 1

collections.Counter has most_common()

✅ Show solution

from collections import Counter
import re

def top_k(text, k):
    words = re.findall(r"[a-z']+", text.lower())
    return [w for w, _ in Counter(words).most_common(k)]

How it works

re.findall(r"[a-z']+", text.lower()) pulls out runs of letters (and apostrophes, so "don't" stays one word) and ignores everything else. That deals with punctuation in one line. Counter counts them, and most_common(k) gives back the top k as (word, count) pairs. The list comprehension keeps just the words.

Making it useful on real text

On real feedback forms, the top words are almost always "the", "a", "is", "and". They're called stop words, and you usually want to drop them:

STOP = {"the", "a", "an", "is", "and", "to", "of", "in", "it", "was"}

def top_k(text, k):
    words = re.findall(r"[a-z']+", text.lower())
    return [w for w, _ in Counter(w for w in words if w not in STOP).most_common(k)]

Note that the pattern only matches English letters. If your text has Hindi or other scripts, use r"\w+" instead.

Complexity: O(n) to count, then picking the top k is fast. most_common uses a heap when you pass k.

Tip: run your code in a local Python environment, Jupyter or Google Colab.