7 Tough RAG Interview Questions ByteDance Asked – Why Most Candidates Fail the First Three

First Question: What are your data sources?

The interviewer expects you to discuss the heterogeneity of sources, not just say “PDF and Markdown”. You must classify sources by structural level (structured, semi‑structured, unstructured) and explain the downstream impact.

Data Type        Example                         Difficulty   Key Challenge
Structured       DB tables, API JSON, Excel      Low          Semantic alignment, field mapping
Semi‑structured  Markdown, HTML, Confluence Wiki Medium       Noise cleaning, structure extraction
Unstructured     PDF, Word, scans, images        High         Format parsing, layout recovery

A good answer mentions internal Wiki, product PDFs, and structured API data, and stresses that each class requires a different preprocessing pipeline.

Second Question: How do you handle redundant or irrelevant information before indexing?

A superficial answer (“we cleaned the data”) is insufficient. A real RAG system uses a five‑layer cleaning pipeline.

Layer 1 – Deduplication

Content‑level deduplication using SimHash or MinHash. Two chunks with a Hamming distance < 3 are considered duplicates.

from simhash import Simhash

def compute_simhash(text: str) -> int:
    tokens = jieba.lcut(text)
    return Simhash(tokens).value

def is_duplicate(hash1: int, hash2: int, threshold: int = 3) -> bool:
    return bin(hash1 ^ hash2).count('1') <= threshold

Keep the most authoritative version (official docs > internal Wiki > personal notes) and record the authority in metadata.

Layer 2 – Noise Filtering

Rule‑based removal of short chunks (< 50 chars), repeated headers/footers, pure numbers or symbols, plus a lightweight classifier that drops chunks with < 30 % lexical content.

Layer 3 – PII Redaction

Combine regex (for fixed‑format data) with NER models to mask phone numbers, ID numbers, internal IPs, etc.

Layer 4 – Encoding Normalisation

Convert all text to UTF‑8, unify full‑width/half‑width characters, and strip invisible characters.

Layer 5 – Quality Scoring

Score each chunk on information density, semantic completeness, and source authority; drop anything below a threshold.

Information density (ratio of content words)

Semantic completeness (does it represent a full knowledge point?)

Authority (official > personal)

Third Question: How do you process PDFs?

PDFs are layout description languages, not plain text. The pipeline distinguishes five content types.

1. Pure Text Paragraphs

Use PyMuPDF (fitz) to extract text blocks with coordinates, then perform column detection for multi‑column documents.

2. Tables

Tables lack semantic tags; you must detect line grids and cell boundaries. Tool matrix:

Tool      Suitable for                Limitation
Camelot   Bordered tables            No border → fail
Tabula    Simple rule‑based tables   Complex merges → fail
PaddleOCR PP‑Structure               Scanned/complex tables → model‑dependent
LLM       Irregular/semantic tables  High cost, slower

Typical workflow: try Camelot → fallback to PP‑Structure OCR → use LLM for the hardest cases. Convert extracted tables to Markdown or JSON before embedding.

3. Images

Discard decorative images. For informational images, run OCR; if no text, use a multimodal model to generate a description and store both the description and the image reference.

4. Headers/Footers

Detect repeated blocks that appear on > 30 % of pages and remove them.

5. Bookmarks & TOC

Extract bookmarks as hierarchical metadata; use them to guide chunk boundaries instead of fixed‑length splits.

"PDF processing must be classified: plain text → layout analysis, tables → structured extraction, images → OCR + multimodal description. Treating everything with a single PyPDF2 read is the main reason RAG quality stalls."

Fourth Question: Do you perform knowledge extraction?

Chunking and knowledge extraction are distinct. Extraction operates on chunks to produce structured entities, relations, and enriched metadata.

Entity Extraction

Identify product names, version numbers, technical terms, etc., using NER models (spaCy, HanLP) plus LLM assistance.

Relation Extraction

Capture links such as "Spring Boot depends on Spring Framework" to enable query expansion.

Metadata Enrichment

Store each chunk with a JSON payload containing IDs, source document, page, section, entities, keywords, version, and ingest timestamp.

{
  "chunk_id": "chunk_0823",
  "content": "Spring Boot 3.0 introduced ...",
  "source_doc": "spring-boot-3-migration-guide.pdf",
  "page": 23,
  "section": "Auto‑configuration changes",
  "entities": ["Spring Boot 3.0", "spring-boot-starter-web"],
  "keywords": ["auto‑configuration", "starter", "migration"],
  "doc_version": "v3.0.1",
  "ingest_time": "2025-06-15T10:30:00Z"
}

LLM function‑calling (e.g., GPT‑4o‑mini) is recommended for cost‑effective extraction.

extraction_prompt = """
Extract from the following text:
{chunk_text}
1. Key entities
2. Relations
3. Core keywords (3‑5)
4. One‑sentence summary
Return JSON.
"""

Fifth Question: How do you handle incremental updates?

Incremental updates require change detection and minimal re‑embedding.

Step 1 – Change Detection

Compute a SHA‑256 hash for each document at ingest time and maintain a registry.

import hashlib

def compute_doc_hash(filepath: str) -> str:
    with open(filepath, 'rb') as f:
        return hashlib.sha256(f.read()).hexdigest()

Periodically scan the source directory, compare hashes, and classify files as new, changed, or deleted.

Step 2 – Minimal Update

Perform document‑level diff: chunk old and new versions, align with SimHash, skip unchanged chunks, replace changed ones, insert new chunks, and soft‑delete removed chunks.

def incremental_update(doc_id: str, new_chunks: list, old_chunks: list):
    # Align using SimHash
    old_hashes = {c.id: compute_simhash(c.text) for c in old_chunks}
    new_hashes = {c.id: compute_simhash(c.text) for c in new_chunks}
    to_add, to_update, to_delete = [], [], []
    for new_chunk in new_chunks:
        matched = False
        for old_id, old_hash in old_hashes.items():
            if is_duplicate(new_hashes[new_chunk.id], old_hash):
                if content_changed_beyond_threshold(new_chunk, old_chunks[old_id]):
                    to_update.append((old_id, new_chunk))
                matched = True
                break
        if not matched:
            to_add.append(new_chunk)
    for old_id in old_hashes:
        if old_id not in [u[0] for u in to_update]:
            to_delete.append(old_id)
    batch_embed_and_insert(to_add)
    batch_embed_and_replace(to_update)
    soft_delete(to_delete)

Step 3 – Embedding Version Management

Store the embedding model version in metadata; when the model changes, perform a full re‑embedding rather than an incremental one.

"If the model changes, we run a full rebuild on a new collection, gray‑switch traffic, verify quality, then cut over."

Sixth Question: How do you resolve conflicting information from different documents?

Conflict handling is a three‑layer strategy.

Layer 1 – Prevention (Authority Ranking)

Assign authority levels (L1 = official, L2 = formal internal, L3 = informal internal, L4 = external) and store them in metadata.

Authority Level   Source
L1 (high)         Official docs, latest specs
L2                Internal formal docs
L3                Internal informal notes
L4 (low)          Blogs, forum posts

Layer 2 – Detection

After retrieval, compare chunks pairwise; if semantic similarity > 0.85 but key entities differ, flag a conflict. An LLM prompt can be used for finer judgment.

conflict_detection_prompt = """
Given the retrieved snippets, determine if they contradict each other.
Snippet A: {chunk_a}
Snippet B: {chunk_b}
If contradictory, point out the conflict.
"""

Layer 3 – Resolution

Four possible actions:

Authority‑first: keep the higher‑authority chunk, downgrade the other.

Recency‑first: prefer the newer version.

LLM fusion: let the LLM generate an answer that cites both versions and notes the difference.

Human escalation: flag critical conflicts for manual review.

def rerank_by_authority(results: list) -> list:
    conflict_groups = detect_conflicts(results)
    for group in conflict_groups:
        group.sort(key=lambda x: x.metadata['authority_level'])
        for i, chunk in enumerate(group):
            if i > 0:
                chunk.score *= 0.3  # demote lower‑authority chunks
    return sorted(results, key=lambda x: x.score, reverse=True)

Seventh Question: Do you have version control and fault‑tolerance for the knowledge base?

Knowledge‑base versioning records a doc_id, filename, version tag, content hash, parent version, ingest time, and status.

doc_id: doc_0823
filename: api-guide.pdf
version: v3.2
content_hash: a3f8b2...
parent_version: v3.1
ingest_time: 2025-06-15T10:30:00Z
status: active

When a new version arrives, the old version is marked superseded rather than deleted, enabling rollback.

Ingestion Workflow

Documents flow through a staging area, automatic quality checks, manual review, and finally official ingestion.

[Upload] → [Staging] → [Auto QC] → [Human Review] → [Official Store]

Auto QC checks for emptiness, encoding issues, quality score, duplication, and sensitive data.

Rollback Mechanism

Three steps: isolate (mark chunks quarantined), rollback to the previous version (set its chunks to active), and assess impact on past queries.

def quarantine_document(doc_id: str, reason: str):
    chunks = get_chunks_by_doc(doc_id)
    for chunk in chunks:
        chunk.metadata['status'] = 'quarantined'
        chunk.metadata['quarantine_reason'] = reason
        chunk.metadata['quarantine_time'] = datetime.now().isoformat()
    vector_store.update_metadata(chunks)

Audit Logging

All ingest, update, delete, and quarantine actions are logged with timestamp, operator, document ID, version, and status.

Time                Operator   Action      Doc ID   Version   Status
2025-06-15 10:30   zhang_san  ingest      doc_0823 v3.2     active
2025-06-15 14:00   li_si      quarantine  doc_0823 v3.2     quarantined
2025-06-15 15:00   li_si      rollback    doc_0823 v3.1     active

What the Interviewer Is Really Looking For

The seven questions form a progressive chain: data source awareness, end‑to‑end cleaning, deep PDF handling, structured knowledge modeling, operational maintenance, conflict robustness, and governance. Candidates who can discuss concrete tools, code snippets, and fallback strategies demonstrate real‑world RAG experience; those who only recite textbook steps do not.

In short, the depth of a RAG project lies in data engineering rigor, not in the fanciest embedding model.