Content revision 4e771eaf0390cc87b6c7c6fd8469f3508f6ceeec7119472e6c1ccf17e2e1e5d2

## Making a flowchart’s decision logic searchable

A Structure-Preserving RAG Ingestion Pipeline
Source ID ragflow-pipeline:mermaid
Original https://www.henryw.me/#/ragflow-pipeline#mermaid

Many client documents used flowchart images to explain a procedure. An answer might depend on following a “Yes” branch through several conditions. OCR could recover the words in each box while losing the arrows that made them a decision. I also tested free-form VLM descriptions, which conveyed the overall process with varying levels of detail. I chose Mermaid, a text-based diagram language that makes each branch explicit through nodes, connecting edges and labels. This let me inspect the individual decision paths. A VLM could convert the image into this representation, making its logic searchable alongside the document’s prose. Rendering the text back into a diagram also gave me a way to compare the reconstructed decisions with the original. In the controlled test, the reconstruction retained every node, edge and label and supported correct answers to both flowchart questions.

Later in the research, I found that MinerU’s built-in VLM already supported converting flowcharts to Mermaid. It could select an appropriate representation for an image, including HTML for tables and Mermaid for flowcharts. This brought the two visual requirements into the same parser and model, fitting the client’s preference to keep the number of models they operated small.