Building the RAG index

The RAG index is a derived artifact: it is rebuilt from source PDFs, not committed. What is committed is the recipe — services/mmore/index_manifest.yaml declares which documents belong in the index, under which file_id, from which URL, and with which checksum.

make mmore-index          # build/refresh on the main service (port 8000)
make mmore-index-eval     # build/refresh on the eval service (port 8001)
make mmore-index-verify   # assert the index answers, without writing

Why this exists

Before the manifest, the evaluation corpus existed only as chunks inside one untracked Milvus Lite database file. The ids engibench_paper and soptx_paper — which the rag_beams2d benchmark depends on — were recorded nowhere but a three-line comment in a shell script, and no command could recreate them. The benchmark was reproducible only on the one machine that happened to hold that file.

The manifest

collection_name: my_docs

documents:
  - file_id: engibench_paper
    path: papers/2508.00831v2.pdf
    url: https://arxiv.org/pdf/2508.00831v2
    sha256: 3f70dc96…
    required_by: [benchmarks/problems/rag_beams2d]

verify:
  - query: "EngiBench API example desired_conds volfrac forcedist beams2d"
    file_ids: [engibench_paper]
    must_contain_any: ["0.7", "0.3"]

The file_ids are load-bearing: renaming one invalidates the RAG benchmark.

Source PDFs are gitignored, so a fresh checkout has none of them. The build script downloads each missing file from its url and verifies the sha256 before indexing. A mismatch is a hard failure, not a warning — indexing a different revision of a paper would silently change what the agent can find.

Verification

The verify block holds probe queries that mirror what the benchmark expects to be retrievable. --verify-only runs them without writing anything, and exits non-zero on failure, so it works as a preflight gate. run_rag_benchmark_suite.sh calls it before doing anything else: that suite takes hours across three models and three RAG conditions, and an empty index would produce a full run of zeros that reads as a model failure rather than a missing index.

Options

Flag	Effect
--only FILE_ID …	Process just these documents
--force	Re-index documents that already exist (via PUT, in place)
--dry-run	Report the plan without contacting the service or writing
--verify	Run the probe queries after building
--verify-only	Run the probe queries only
--mmore-url URL	Override MMORE_RAG_URL

Re-running without --force is a no-op for documents that are already indexed, so it is safe to call repeatedly.

Ad-hoc ingestion

The manifest covers the documents the project requires. For bulk personal collections, scripts/import_local_papers.py walks PAPERS_SOURCE_DIR and derives ids from relative paths, tracking what it has already imported. Both paths share the same retry and duplicate handling.

This is not the published index

The index this manifest builds is a clean two-document corpus. The index used for the published RAG results held seven documents and 267 chunks, including an unrelated arXiv paper (100 chunks) and the Andreassen 88-line paper (53 chunks).

That difference matters more than it looks: search_documents retrieves without a file_ids filter, so every one of those chunks competed for the top-k=5 slots. To reproduce published numbers, restore the archived database described in the frozen v1 stack rather than rebuilding from this manifest.