RAG Tool Curator docs v0.1.0
Practitioner Notes

Reusability Guide

The retrieval algorithm doesn't know anything about universities. It operates on (name, description) pairs, a query string, and a language-detection heuristic — everything domain-specific lives outside both packages.

What carries over unchanged

Both tool_curator/ and multilingual_normalizer/ — the curator interface, the hybrid retrieval engine, the suggestion validator, and the normalizer’s interface/fast-path/Gemini backend are all domain-agnostic by construction and require zero edits to point at a new domain. Copy them in (or pip install them) and go.

example/orchestrator/hallucination_guard.py and the MCP stdio transport patch also carry over as-is if your new project happens to also run FastMCP over stdio — neither has any domain knowledge baked in.

What needs small, per-domain edits

Nothing in the plug-ins themselves — but the configuration files that feed them are inherently per-domain:

  • domain_synonyms.json — your tools’ vocabulary, not the university’s.
  • dependency_rules.json / equivalences.json — your workflows’ companion-tool relationships, not the university’s.
  • The eval spreadsheet — your own labeled test queries.

What gets rebuilt from scratch

Everything specific to the reference deployment itself: the mock database, the 8 MCP servers, the persona/role policy, the mutation guard’s tool list, and the system prompt. Role filtering still has to run before either plug-in touches a query, regardless of domain — that responsibility never moves into the plug-ins.

Data contract

The curator needs only (name, description) pairs — these can come from an MCP list_tools() call, an OpenAI tool spec, a LangChain tool registry, or a plain database of tool metadata. The normalizer needs only a query string. Neither cares how you got there.

An illustrative (not validated) mapping — university → banking

Piece University Banking
Identity fields student_id, faculty_id customer_id, account_id
Example dependency rule get_exam_eligibility → attendance + fees initiate_fund_transfer → balance + beneficiary verification + limits
Example synonym entry submit_grievance: “raise complaint, lodge issue” block_debit_card: “lost card, stolen card, freeze card”
High-risk action tools submit marks, book a lab, file a grievance transfer money, close an account, block a card

Healthcare maps similarly: patient_id/doctor_id as identity fields, prescribe_medication depending on patient history and allergy data. These mappings are illustrative, not measured — see the honesty note below.

Sizing top_k

  • 6–8 for small tool sets where tools are easily distinguished from each other.
  • 10–15 for larger sets with many similar-sounding tools (the reference deployment’s own case).
  • 10 + dependency expansion for high-risk workflows where a companion tool’s data is structurally required, not just topically related.

Embedding model choice

BAAI/bge-small-en-v1.5 was the validated default throughout. bge-base-en-v1.5 trades some speed for better paraphrase accuracy on harder domains. all-MiniLM-L6-v2 was noticeably weaker in informal comparisons during development and was never used for any reported result — that’s a disclosed gap, not a benchmarked recommendation against it.

Minimal end-to-end example

from tool_curator import get_curator, load_tool_context

curator = get_curator(mode="rag", top_k=10, index_dir=".tool_index")

tools = [("get_account_balance", "Returns account balance for a customer account.")]
curator.build_index(tools, extra_context=load_tool_context("domain_synonyms.json"))

selected_tools = curator.select("Can I send money to Rahul?", tools)

build_index() mutates internal state and writes the disk cache; it has no return value. select() returns ranked (name, description) tuples, typically 10–18 items depending on anchor and dependency additions — serialize into whatever your target LLM’s tool-spec format requires.

Reusing only the algorithm underperforms what's reported here

FAISS, BM25, and RRF don't care what they're indexing — but the 94.8% accuracy score depends on four things working together: good tool names, contract-style descriptions, a populated synonym file, and dependency rules reflecting your actual workflows. Porting the retrieval code without doing the other three is likely to land well below these numbers.