Advanced Retrieval: Filters, MMR and Late Interaction

Metadata Pre-Filtering

Narrow the candidate set before ranking so selective filters do not waste distance computation. Store source, date, author and type on every chunk.

# ChromaDB
hits = kb.query(query_texts=[question], n_results=5,
                where={"year": {"$gte": 2024}})

# pgvector
# SELECT * FROM chunks WHERE year >= 2024
# ORDER BY embedding <=> '[0.1, ...]' LIMIT 5;

MMR for Diversity

Maximal Marginal Relevance balances similarity with diversity so the top results are not ten near-copies of one paragraph.

def mmr_selected(candidates, query_vec, selected, lam=0.7):
    best = None
    best_score = -1.0
    for c in candidates:
        sim = cosine(query_vec, c["vec"])
        if selected:
            similar_to_selected = max(
                cosine(c["vec"], s) for s in selected
            )
        else:
            similar_to_selected = 0.0
        score = lam * sim - (1 - lam) * similar_to_selected
        if score > best_score:
            best, best_score = c, score
    return best

Late Interaction

Late-interaction models such as ColBERT compare token by token instead of one vector per text. They capture finer relevance and rank far better, at higher compute cost.

When to Use

• Filters - Always, when metadata exists.
• MMR - Summaries or lists needing variety.
• Late interaction - Hard domains when plain dense ranking underperforms.

Key Points

• Filter early when filters are selective.
• MMR swaps pure similarity for diversity.
• Late interaction gives finer matching for hard cases.
• Combine techniques for the quality level you need.