Content revision ea31a3343577b30cee4337cf2fef2b0864a5741341f27598e9108766c486fb2d

## Bringing the four requirements into one pipeline

A Structure-Preserving RAG Ingestion Pipeline
Source ID ragflow-pipeline:what-i-tested-before-recommending-it
Original https://www.henryw.me/#/ragflow-pipeline#what-i-tested-before-recommending-it

The recommended architecture combined MinerU with its built-in VLM, a post-parsing reference linker, and LlamaIndex. Parsing recovered document continuity and represented tables and diagrams in structured text. The linker attached referenced notes and remarks before LlamaIndex formed chunks for retrieval. An embedding model represented those chunks and the question as numerical vectors, allowing passages with related meanings to be found even when their wording differed. The answer model then received the retrieved evidence with its relationships intact. I integrated these components in a proof of concept and followed the controlled document through the full path, checking what reached the answer model after each processing choice. This exposed losses that a successful parsing preview had concealed and gave the client an end-to-end basis for choosing the tools.

I separated GPU-backed parsing and visual interpretation from CPU orchestration, reference linking and indexing, then compared on-premises and private-cloud deployment. On-demand processing could release resources between ingestion jobs. Starting a new job would then take time to load the model. Keeping the service running traded that delay for ongoing cost. These choices let the client assess the pipeline against its infrastructure, ingestion frequency and maintenance capacity. 06 Reflection