Understanding NDCG@k (Normalized Discounted Cumulative Gain)
Think of it like a teacher grading a “Top 10” list — not just checking if you picked the right items, but also whether you placed the best ones at the top.
1. Breakdown of the Acronym (Plain English)
G — Gain
The “score” or points for a single result.
Example:
Searching for Star Wars:
- Actual Star Wars movie → High Gain (3 points)
- Star Trek movie → Zero Gain (0 points)
C — Cumulative
The total score after adding all gains.
D — Discounted
A penalty for placing good results too far down the list.
A great result at #1 is far more valuable than the same result at #10, because fewer users scroll that far.
N — Normalized
Converts the score into a 0–1 scale by comparing it to a perfect ranking.
If:
- Your score = 8
- Perfect score = 10
→ NDCG = 0.8
@k — Cutoff
We only evaluate the top k results.
- NDCG@5 → Grade only the top 5 results.
2. A Simple Analogy: The Music Playlist
Imagine asking a DJ (the AI) to create a Top 5 Rock Songs playlist (k = 5).
Scoring Rules
- Gain:
- 3 stars = Perfect
- 2 stars = Okay
- 0 stars = Bad
- Discounting: The first song matters most.
Scenario A — The “Okay” Playlist
- Song 1: Nickelback (2 stars) ← High-value spot wasted
- Song 2: Led Zeppelin (3 stars) ← Great song, buried too low
- Song 3: Beethoven (0 stars)
- …
Scenario B — The “Ideal” Playlist
- Song 1: Led Zeppelin (3 stars) ← Best song in the best position
- Song 2: Nickelback (2 stars)
- Song 3: Beethoven (0 stars)
- …
Even though both playlists contain the same songs, Scenario B gets a much higher NDCG because the best item appears at the top.
3. How to Calculate NDCG@k (Step-by-Step)
Suppose we have 3 documents with relevance scores: 3, 2, 1
We want to calculate NDCG@3.
Step 1 — Actual Score (DCG)
Assume your system ranks them: 2, 1, 3
| Position | Relevance | Discount | Contribution |
|---|---|---|---|
| 1 | 2 | No discount | 2.0 |
| 2 | 1 | Small discount | 0.6 |
| 3 | 3 | Big discount | 1.5 |
Total DCG = 4.1
Step 2 — Perfect Score (IDCG)
Correct order: 3, 2, 1
| Position | Relevance | Discount | Contribution |
|---|---|---|---|
| 1 | 3 | No discount | 3.0 |
| 2 | 2 | Small discount | 1.2 |
| 3 | 1 | Big discount | 0.5 |
Total IDCG = 4.7
Step 3 — Normalize (NDCG)
[ \text{NDCG} = \frac{\text{DCG}}{\text{IDCG}} = \frac{4.1}{4.7} = \mathbf{0.87} ]
The Mathematical Magic of NDCG
NDCG turns a vague idea like “ranking quality” into a precise number between 0 and 1.
It does this in three steps.
Master Formula
[ NDCG = \frac{DCG}{IDCG} ]
It looks simple, but we must compute:
- DCG → Your actual score
- IDCG → The perfect possible score
Step 1: Calculate DCG (Your Actual Score)
DCG = Discounted Cumulative Gain
It adds up relevance scores while “punishing” results that appear too far down the ranking.
The score contributed by the result at position i:
[ \frac{rel_i}{\log_2(i+1)} ]
Where:
- relᵢ — relevance score (e.g., 3 = Perfect, 0 = Bad)
- log₂(i+1) — discount factor; the deeper the position, the bigger the penalty
Example: Top-3 Ranking (k = 3)
Relevance scores of the documents: 3 (Perfect), 2 (Good), 1 (Okay)
Your AI produced the ranking: [2, 3, 1]
(“Good” came first, “Perfect” came second.)
DCG Calculation
Position 1 (Score = 2)
[ \frac{2}{\log_2(1+1)} = \frac{2}{1} = \mathbf{2.0} ]
The first position always gives full value because
(\log_2(2)=1)
Position 2 (Score = 3)
[ \frac{3}{\log_2(2+1)} = \frac{3}{1.58} \approx \mathbf{1.89} ]
We lost value here — a relevance of 3 should have been worth 3.0,
but the position penalty reduced it to 1.89.
Position 3 (Score = 1)
[ \frac{1}{\log_2(3+1)} = \frac{1}{2} = \mathbf{0.5} ]
Total DCG (Your Score)
[ 2.0 + 1.89 + 0.5 = \mathbf{4.39} ]
Step 2: Calculate IDCG (Perfect Score)
IDCG = Ideal Discounted Cumulative Gain
This is what you would score with a perfect ranking.
Ideal ranking: [3, 2, 1]
IDCG Calculation
Position 1 (Score = 3)
[ \frac{3}{\log_2(2)} = \frac{3}{1} = \mathbf{3.0} ]
Position 2 (Score = 2)
[ \frac{2}{\log_2(3)} = \frac{2}{1.58} \approx \mathbf{1.26} ]
Position 3 (Score = 1)
[ \frac{1}{\log_2(4)} = \frac{1}{2} = \mathbf{0.5} ]
Total IDCG (Perfect Score)
[ 3.0 + 1.26 + 0.5 = \mathbf{4.76} ]
Step 3: Normalize (Compute NDCG)
Compare your DCG to the perfect possible DCG:
[ NDCG = \frac{4.39}{4.76} = \mathbf{0.92} ]
Final Result
Your NDCG score = 0.92
This means:
Your ranking is 92% as good as the perfect ranking.
Summary
- High NDCG (close to 1.0): The ranking puts the best items at the top.
- Low NDCG (close to 0.0): The ranking buries or misses the most relevant results.