Production RAG with three execution modes

The problem

Business users ask questions whose answers are spread across reports, spreadsheets, slides and PDFs. Some questions need a fast lookup. Others need a careful search over many documents. One fixed pipeline is too slow for the first kind and too shallow for the second.

The approach

The system indexes 11+ document formats and offers three execution modes:

1. Fast. One retrieval pass and one answer.
2. Thorough. A wider search with more context.
3. Agentic. An agent plans several searches and combines the results.

Each mode uses the same retrieval core:

• Hybrid search. Semantic search and keyword search run together, and rank fusion merges the two lists.
• Self-hosted cross-encoder reranking. The reranker runs on our own hardware, so it adds no per-query API cost.
• End-to-end tracing. Each answer can be traced back through retrieval, reranking and generation.

The result

For large organizations, index-aware pre-filtering narrows the candidate set before the expensive steps run, and it cut query latency by 50–70%.