RAG Benchmarking
Version v1.0.2Plug in any RAG system (LangChain, LlamaIndex, or custom) and benchmark it against classic and agentic-era metrics. Faithfulness, answer relevancy, retrieval precision, and four agentic metrics for multi-step agents. Measured faithfulness of 0.958 on the 50-sample golden dataset.
Apache 2.0 · zero telemetry · source Regulation (EU) 2024/1689 (Article 15).
Why RAG Benchmarking exists
Accuracy, robustness and cybersecurity
Article 15 applies from 2 December 2027 for high-risk AI systems under Annex III. The original 2 August 2026 date was moved by the Digital Omnibus, Regulation (EU) 2026/1744, published in the Official Journal on 24 July 2026 and in force since 27 July 2026. Providers must declare accuracy metrics in the instructions for use and demonstrate consistent performance across the lifecycle. Non-compliance via the Article 16 provider-obligation chain is sanctionable up to €15M or 3% of global annual turnover under Article 99(4). For RAG-based high-risk systems, "appropriate accuracy" is not a self-asserted figure. It is a metric declared on the label and defensible against post-market evidence.
Other AiExponent tools