Skip to content

ir_axioms

Axiomatic constraints for information retrieval and retrieval-augmented generation

ir_axioms is a Python framework for experimenting with axioms in information retrieval in a declarative way. It includes reference implementations of many commonly used traditional retrieval axioms as well as axioms for retrieval-augmented generation (RAG) [Bondarenko et al., 2022].

Get started Browse axioms View on GitHub View on PyPI

The library is well integrated with the PyTerrier framework and supports Pyserini retrieval indices too. Use-cases of ir_axioms include search-result re-ranking, analyses of (neural) retrieval systems, and analyses of generated RAG answers.

from ir_axioms.axiom import TFC1, PROX1, ORIG
from ir_axioms.model import Query, Document

query = Query(id="q1", text="axiomatic information retrieval")
documents = [
    Document(id="d1", text="Retrieval axioms formalize constraints for retrieval models.", rank=1),
    Document(id="d2", text="Axiomatic information retrieval: axiomatic constraints guide retrieval.", rank=2),
]

axiom = (TFC1() & PROX1()) | ORIG()  # (1)!
axiom.preferences(input=query, outputs=documents)  # (2)!
axiom.rerank_kwiksort(input=query, ranking=documents)  # (3)!
  1. Combine axioms declaratively: TFC1 and PROX1 must agree, otherwise fall back to the original ranking.
  2. Compute the pairwise preference matrix for all documents.
  3. Re-rank the documents with KwikSort based on the axiom's preferences.

Features

  • Retrieval axioms

    Reference implementations of 30+ retrieval axioms, from TFC1 to PROX5, with documented deviations from their original formulations.

    Retrieval axioms

  • RAG axioms [Merker et al., 2025]

    Axioms for retrieval-augmented generation that address coherence, coverage, consistency, correctness, and clarity of generated responses.

    RAG axioms

  • PyTerrier integration

    Re-rank with KwikSort, compute preference features for learning to rank, and run axiomatic experiments directly in PyTerrier pipelines.

    Integrations

  • Walkthroughs

    Step-by-step tutorials on re-ranking, post-hoc analyses of runs and qrels, and evaluating RAG responses.

    Walkthroughs

When using ir_axioms in your research, consider citing our SIGIR 2022 paper about the framework [Bondarenko et al., 2022] or our ICTIR 2025 paper about the RAG axioms [Merker et al., 2025].

References

Alexander Bondarenko, Maik Fröbe, Jan Heinrich Reimer, Benno Stein, Michael Völske, and Matthias Hagen. Axiomatic Retrieval Experimentation with ir_axioms. In SIGIR '22: The 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, Madrid, Spain, July 11-15, 2022, pages 3131–3140, 2022. ACM. [] [doi] 
@inproceedings{bondarenko2022axiomatic,
  author       = {Alexander Bondarenko and
                  Maik Fr{\"{o}}be and
                  Jan Heinrich Reimer and
                  Benno Stein and
                  Michael V{\"{o}}lske and
                  Matthias Hagen},
  title        = {Axiomatic Retrieval Experimentation with ir{\_}axioms},
  booktitle    = {{SIGIR} '22: The 45th International {ACM} {SIGIR} Conference on Research
                  and Development in Information Retrieval, Madrid, Spain, July 11-15, 2022},
  pages        = {3131--3140},
  publisher    = {{ACM}},
  year         = {2022},
  url          = {https://doi.org/10.1145/3477495.3531743},
  doi          = {10.1145/3477495.3531743}
}
Jan Heinrich Merker, Maik Fröbe, Benno Stein, Martin Potthast, and Matthias Hagen. Axioms for Retrieval-Augmented Generation. In Proceedings of the 2025 International ACM SIGIR Conference on Innovative Concepts and Theories in Information Retrieval, ICTIR 2025, Padua, Italy, July 18, 2025, pages 67–77, 2025. ACM. [] [doi] 
@inproceedings{merker2025axioms,
  author       = {Jan Heinrich Merker and
                  Maik Fr{\"{o}}be and
                  Benno Stein and
                  Martin Potthast and
                  Matthias Hagen},
  title        = {Axioms for Retrieval-Augmented Generation},
  booktitle    = {Proceedings of the 2025 International {ACM} {SIGIR} Conference on
                  Innovative Concepts and Theories in Information Retrieval, {ICTIR}
                  2025, Padua, Italy, July 18, 2025},
  pages        = {67--77},
  publisher    = {{ACM}},
  year         = {2025},
  url          = {https://doi.org/10.1145/3731120.3744601},
  doi          = {10.1145/3731120.3744601}
}