Content revision 4e771eaf0390cc87b6c7c6fd8469f3508f6ceeec7119472e6c1ccf17e2e1e5d2

## A Structure-Preserving RAG Ingestion Pipeline

Builds
Source ID projects:pwcRag
Original https://www.henryw.me/#/projects?to=pwcRag

The architectural risk appeared before retrieval began. If ingestion separated a passage from the note it refers to or reduced a flowchart to disconnected text, no later ranking or generation step could reliably restore the missing relationship.

MinerU rejoined paragraphs and tables across page breaks. Its OCR-based path recovered merged cells but missed nested tables in the tests. The built-in visual-language model recovered both as HTML. I then found that subsequent processing in the tested RAGFlow path discarded those tags, while LlamaIndex preserved the structure through indexing, retrieval and answer generation.

Cross-page footnotes and remarks needed explicit linking. I wrote a Python linker that used reference labels and document context to attach each note to its citing text before chunking. For flowchart images, I chose Mermaid to represent nodes and branches as searchable text that could be rendered back for inspection. Later research showed that MinerU’s built-in VLM already supported this conversion, bringing tables and flowcharts into the same parser and model.

A controlled five-page document let me repeat the checks across the full pipeline. The linker resolved all 13 cross-page references, and the reconstructed flowchart preserved every node, edge and label and supported correct answers to both test questions. I used this evidence alongside deployment and operating-cost analysis to develop the client recommendation, including its preference to operate fewer AI models.