Skip to content

Axioms

Catalog of all axioms implemented in ir_axioms

ir_axioms implements retrieval axioms that compare documents for a query, RAG axioms that compare generated responses for a request, and utility axioms that serve as reference points and building blocks. For a broader overview of axiomatic thinking in information retrieval, including constraints that are not (yet) implemented here, see related axioms from the literature.

Each axiom implementation is tested, featuring an extensive test suite of over 400 unit tests.

To learn how to compute preferences with these axioms and combine them, see the quickstart and core concepts; for the full API, see the API reference.

All axioms can be imported from ir_axioms.axiom, either by their class name (e.g., Tfc1Axiom) or by their short alias (e.g., TFC1).

Retrieval Axioms

Axiom Intuition Objective Source
TFC1 Prefer documents with more query term occurrences. Term frequency [Fang et al., 2004]
TFC3 Prefer documents with occurrences of more query terms. Term frequency [Fang et al., 2011]
M-TDC, LEN-M-TDC Prefer documents with more discriminative query terms. Term frequency [Shi et al., 2005]
LNC1 Penalize longer documents for non-relevant terms. Document length [Fang et al., 2004]
TF-LNC Reward additional query terms more than document length is penalized. Document length [Fang et al., 2004]
LB1 Do not override the term presence–absence gap with length normalization. Lower-bounding TF [Lv and Zhai, 2011]
REG, ANTI-REG, ASPECT-REG Prefer documents covering more different query aspects. Query aspects [Zheng and Fang, 2010]
AND, LEN-AND, M-AND, LEN-M-AND Prefer documents containing all query terms. Query aspects [Wu and Fang, 2012]
DIV, LEN-DIV Prefer documents that are less redundant with the query. Query aspects [Gollapudi and Sharma, 2009]
STMC1 Prefer documents with terms more similar to query terms. Semantic similarity [Fang and Zhai, 2006]
STMC2 Do not reward similar terms more than exact matches. Semantic similarity [Fang and Zhai, 2006]
PROX1 Prefer documents with shorter distance between query term pairs. Term proximity [Hagen et al., 2016]
PROX2 Prefer documents with earlier query term occurrences. Term proximity [Hagen et al., 2016]
PROX3 Prefer documents where the query occurs earlier as a phrase. Term proximity [Hagen et al., 2016]
PROX4 Prefer documents that contain all query terms in a shorter substring. Term proximity [Hagen et al., 2016]
PROX5 Prefer documents where the query terms are closer together on average. Term proximity [Hagen et al., 2016]
ArgUC Prefer more argumentative units. Argumentativeness [Bondarenko et al., 2018; Bondarenko et al., 2019]
QTArg Prefer more query terms in argumentative units. Argumentativeness [Bondarenko et al., 2018; Bondarenko et al., 2019]
QTPArg Prefer earlier query terms in argumentative units. Argumentativeness [Bondarenko et al., 2018; Bondarenko et al., 2019]
aSL Prefer passages with 12–20 words per sentence. Argumentativeness [Bondarenko et al., 2018; Bondarenko et al., 2019]

Not implemented, as they do not induce practically usable pairwise preferences: TFC2, TDC, LNC2, and LB2.

RAG Axioms

Axiom Intuition Dimension Source
COH1 Prefer responses with less variance in avg. word length across sentences. Coherence [Merker et al., 2025]
COH2 Prefer responses with subject–verb pairs closer together. Coherence [Merker et al., 2025]
COV1 Prefer responses with more extracted aspects. Coverage [Merker et al., 2025]
COV2 Prefer responses with less redundant extracted aspects. Coverage [Merker et al., 2025]
COV3 Prefer responses with more sentences covering aspects from the query. Coverage [Merker et al., 2025]
CONS1 Prefer responses with more sentences covering aspects from the context. Consistency [Merker et al., 2025]
CONS2 Prefer responses with higher text overlap with contexts. Consistency [Merker et al., 2025]
CONS3 Penalize aspects mentioned in contradictory phrases. Consistency [Merker et al., 2025]
CORR1 Prefer responses with more sentences referencing sources. Correctness [Merker et al., 2025]
CLAR1 Prefer responses with fewer grammar errors. Clarity [Merker et al., 2025]
CLAR2 Prefer responses with better readability. Clarity [Merker et al., 2025]
GEN_* Retrieval axioms applied to generated responses. — [Merker et al., 2025]

Utility Axioms

Axiom Description Source
ORIG Follow the original ranking (ranks or scores). [Hagen et al., 2016]
ORACLE Follow human relevance judgments. [Bondarenko et al., 2022]
NOP, RANDOM, GT, LT, DIST Baselines and helpers. —
Arithmetic axioms Sum, product, vote, conjunction, cascade, negation, normalization. —

References

Alexander Bondarenko, Michael Völske, Alexander Panchenko, Chris Biemann, Benno Stein, and Matthias Hagen. Webis at TREC 2018: Common Core Track. In Proceedings of the Twenty-Seventh Text REtrieval Conference, TREC 2018, Gaithersburg, Maryland, USA, November 14-16, 2018, 2018. National Institute of Standards and Technology (NIST). [] 
@inproceedings{bondarenko2018webis,
  author       = {Alexander Bondarenko and
                  Michael V{\"{o}}lske and
                  Alexander Panchenko and
                  Chris Biemann and
                  Benno Stein and
                  Matthias Hagen},
  title        = {Webis at {TREC} 2018: Common Core Track},
  booktitle    = {Proceedings of the Twenty-Seventh Text REtrieval Conference, {TREC}
                  2018, Gaithersburg, Maryland, USA, November 14-16, 2018},
  series       = {{NIST} Special Publication},
  volume       = {500-331},
  publisher    = {National Institute of Standards and Technology {(NIST)}},
  year         = {2018},
  url          = {https://trec.nist.gov/pubs/trec27/papers/Webis-CC.pdf}
}
Alexander Bondarenko, Maik Fröbe, Vaibhav Kasturia, Michael Völske, Benno Stein, and Matthias Hagen. Webis at TREC 2019: Decision Track. In Proceedings of the Twenty-Eighth Text REtrieval Conference, TREC 2019, Gaithersburg, Maryland, USA, November 13-15, 2019, 2019. National Institute of Standards and Technology (NIST). [] 
@inproceedings{bondarenko2019webis,
  author       = {Alexander Bondarenko and
                  Maik Fr{\"{o}}be and
                  Vaibhav Kasturia and
                  Michael V{\"{o}}lske and
                  Benno Stein and
                  Matthias Hagen},
  title        = {Webis at {TREC} 2019: Decision Track},
  booktitle    = {Proceedings of the Twenty-Eighth Text REtrieval Conference, {TREC}
                  2019, Gaithersburg, Maryland, USA, November 13-15, 2019},
  series       = {{NIST} Special Publication},
  volume       = {1250},
  publisher    = {National Institute of Standards and Technology {(NIST)}},
  year         = {2019},
  url          = {https://trec.nist.gov/pubs/trec28/papers/Webis.D.pdf}
}
Alexander Bondarenko, Maik Fröbe, Jan Heinrich Reimer, Benno Stein, Michael Völske, and Matthias Hagen. Axiomatic Retrieval Experimentation with ir_axioms. In SIGIR '22: The 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, Madrid, Spain, July 11-15, 2022, pages 3131–3140, 2022. ACM. [] [doi] 
@inproceedings{bondarenko2022axiomatic,
  author       = {Alexander Bondarenko and
                  Maik Fr{\"{o}}be and
                  Jan Heinrich Reimer and
                  Benno Stein and
                  Michael V{\"{o}}lske and
                  Matthias Hagen},
  title        = {Axiomatic Retrieval Experimentation with ir{\_}axioms},
  booktitle    = {{SIGIR} '22: The 45th International {ACM} {SIGIR} Conference on Research
                  and Development in Information Retrieval, Madrid, Spain, July 11-15, 2022},
  pages        = {3131--3140},
  publisher    = {{ACM}},
  year         = {2022},
  url          = {https://doi.org/10.1145/3477495.3531743},
  doi          = {10.1145/3477495.3531743}
}
Hui Fang, Tao Tao, and ChengXiang Zhai. A formal study of information retrieval heuristics. In SIGIR 2004: Proceedings of the 27th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Sheffield, UK, July 25-29, 2004, pages 49–56, 2004. ACM. [] [doi] 
@inproceedings{fang2004formal,
  author       = {Hui Fang and
                  Tao Tao and
                  ChengXiang Zhai},
  title        = {A formal study of information retrieval heuristics},
  booktitle    = {{SIGIR} 2004: Proceedings of the 27th Annual International {ACM} {SIGIR}
                  Conference on Research and Development in Information Retrieval,
                  Sheffield, UK, July 25-29, 2004},
  pages        = {49--56},
  publisher    = {{ACM}},
  year         = {2004},
  url          = {https://doi.org/10.1145/1008992.1009004},
  doi          = {10.1145/1008992.1009004}
}
Hui Fang and ChengXiang Zhai. Semantic term matching in axiomatic approaches to information retrieval. In SIGIR 2006: Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Seattle, Washington, USA, August 6-11, 2006, pages 115–122, 2006. ACM. [] [doi] 
@inproceedings{fang2006semantic,
  author       = {Hui Fang and
                  ChengXiang Zhai},
  title        = {Semantic term matching in axiomatic approaches to information retrieval},
  booktitle    = {{SIGIR} 2006: Proceedings of the 29th Annual International {ACM} {SIGIR}
                  Conference on Research and Development in Information Retrieval,
                  Seattle, Washington, USA, August 6-11, 2006},
  pages        = {115--122},
  publisher    = {{ACM}},
  year         = {2006},
  url          = {https://doi.org/10.1145/1148170.1148193},
  doi          = {10.1145/1148170.1148193}
}
Hui Fang, Tao Tao, and ChengXiang Zhai. Diagnostic Evaluation of Information Retrieval Models. ACM Trans. Inf. Syst., 29 (2) : 7:1–7:42, 2011. [] [doi] 
@article{fang2011diagnostic,
  author       = {Hui Fang and
                  Tao Tao and
                  ChengXiang Zhai},
  title        = {Diagnostic Evaluation of Information Retrieval Models},
  journal      = {{ACM} Trans. Inf. Syst.},
  volume       = {29},
  number       = {2},
  pages        = {7:1--7:42},
  year         = {2011},
  url          = {https://doi.org/10.1145/1961209.1961210},
  doi          = {10.1145/1961209.1961210}
}
Sreenivas Gollapudi and Aneesh Sharma. An axiomatic approach for result diversification. In Proceedings of the 18th International Conference on World Wide Web, WWW 2009, Madrid, Spain, April 20-24, 2009, pages 381–390, 2009. ACM. [] [doi] 
@inproceedings{gollapudi2009axiomatic,
  author       = {Sreenivas Gollapudi and
                  Aneesh Sharma},
  title        = {An axiomatic approach for result diversification},
  booktitle    = {Proceedings of the 18th International Conference on World Wide Web,
                  {WWW} 2009, Madrid, Spain, April 20-24, 2009},
  pages        = {381--390},
  publisher    = {{ACM}},
  year         = {2009},
  url          = {https://doi.org/10.1145/1526709.1526761},
  doi          = {10.1145/1526709.1526761}
}
Matthias Hagen, Michael Völske, Steve Göring, and Benno Stein. Axiomatic Result Re-Ranking. In Proceedings of the 25th ACM International Conference on Information and Knowledge Management, CIKM 2016, Indianapolis, IN, USA, October 24-28, 2016, pages 721–730, 2016. ACM. [] [doi] 
@inproceedings{hagen2016axiomatic,
  author       = {Matthias Hagen and
                  Michael V{\"{o}}lske and
                  Steve G{\"{o}}ring and
                  Benno Stein},
  title        = {Axiomatic Result Re-Ranking},
  booktitle    = {Proceedings of the 25th {ACM} International Conference on Information
                  and Knowledge Management, {CIKM} 2016, Indianapolis, IN, USA, October
                  24-28, 2016},
  pages        = {721--730},
  publisher    = {{ACM}},
  year         = {2016},
  url          = {https://doi.org/10.1145/2983323.2983704},
  doi          = {10.1145/2983323.2983704}
}
Yuanhua Lv and ChengXiang Zhai. Lower-bounding term frequency normalization. In Proceedings of the 20th ACM Conference on Information and Knowledge Management, CIKM 2011, Glasgow, United Kingdom, October 24-28, 2011, pages 7–16, 2011. ACM. [] [doi] 
@inproceedings{lv2011lower,
  author       = {Yuanhua Lv and
                  ChengXiang Zhai},
  title        = {Lower-bounding term frequency normalization},
  booktitle    = {Proceedings of the 20th {ACM} Conference on Information and Knowledge
                  Management, {CIKM} 2011, Glasgow, United Kingdom, October 24-28, 2011},
  pages        = {7--16},
  publisher    = {{ACM}},
  year         = {2011},
  url          = {https://doi.org/10.1145/2063576.2063584},
  doi          = {10.1145/2063576.2063584}
}
Jan Heinrich Merker, Maik Fröbe, Benno Stein, Martin Potthast, and Matthias Hagen. Axioms for Retrieval-Augmented Generation. In Proceedings of the 2025 International ACM SIGIR Conference on Innovative Concepts and Theories in Information Retrieval, ICTIR 2025, Padua, Italy, July 18, 2025, pages 67–77, 2025. ACM. [] [doi] 
@inproceedings{merker2025axioms,
  author       = {Jan Heinrich Merker and
                  Maik Fr{\"{o}}be and
                  Benno Stein and
                  Martin Potthast and
                  Matthias Hagen},
  title        = {Axioms for Retrieval-Augmented Generation},
  booktitle    = {Proceedings of the 2025 International {ACM} {SIGIR} Conference on
                  Innovative Concepts and Theories in Information Retrieval, {ICTIR}
                  2025, Padua, Italy, July 18, 2025},
  pages        = {67--77},
  publisher    = {{ACM}},
  year         = {2025},
  url          = {https://doi.org/10.1145/3731120.3744601},
  doi          = {10.1145/3731120.3744601}
}
Shuming Shi, Ji-Rong Wen, Qing Yu, Ruihua Song, and Wei-Ying Ma. Gravitation-based model for information retrieval. In SIGIR 2005: Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Salvador, Brazil, August 15-19, 2005, pages 488–495, 2005. ACM. [] [doi] 
@inproceedings{shi2005gravitation,
  author       = {Shuming Shi and
                  Ji{-}Rong Wen and
                  Qing Yu and
                  Ruihua Song and
                  Wei{-}Ying Ma},
  title        = {Gravitation-based model for information retrieval},
  booktitle    = {{SIGIR} 2005: Proceedings of the 28th Annual International {ACM} {SIGIR}
                  Conference on Research and Development in Information Retrieval,
                  Salvador, Brazil, August 15-19, 2005},
  pages        = {488--495},
  publisher    = {{ACM}},
  year         = {2005},
  url          = {https://doi.org/10.1145/1076034.1076117},
  doi          = {10.1145/1076034.1076117}
}
Hao Wu and Hui Fang. Relation Based Term Weighting Regularization. In Advances in Information Retrieval - 34th European Conference on IR Research, ECIR 2012, Barcelona, Spain, April 1-5, 2012. Proceedings, pages 109–120, 2012. Springer. [] [doi] 
@inproceedings{wu2012relation,
  author       = {Hao Wu and
                  Hui Fang},
  title        = {Relation Based Term Weighting Regularization},
  booktitle    = {Advances in Information Retrieval - 34th European Conference on {IR}
                  Research, {ECIR} 2012, Barcelona, Spain, April 1-5, 2012. Proceedings},
  series       = {Lecture Notes in Computer Science},
  volume       = {7224},
  pages        = {109--120},
  publisher    = {Springer},
  year         = {2012},
  url          = {https://doi.org/10.1007/978-3-642-28997-2\_10},
  doi          = {10.1007/978-3-642-28997-2\_10}
}
Wei Zheng and Hui Fang. Query Aspect Based Term Weighting Regularization in Information Retrieval. In Advances in Information Retrieval, 32nd European Conference on IR Research, ECIR 2010, Milton Keynes, UK, March 28-31, 2010. Proceedings, pages 344–356, 2010. Springer. [] [doi] 
@inproceedings{zheng2010query,
  author       = {Wei Zheng and
                  Hui Fang},
  title        = {Query Aspect Based Term Weighting Regularization in Information Retrieval},
  booktitle    = {Advances in Information Retrieval, 32nd European Conference on {IR}
                  Research, {ECIR} 2010, Milton Keynes, UK, March 28-31, 2010. Proceedings},
  series       = {Lecture Notes in Computer Science},
  volume       = {5993},
  pages        = {344--356},
  publisher    = {Springer},
  year         = {2010},
  url          = {https://doi.org/10.1007/978-3-642-12275-0\_31},
  doi          = {10.1007/978-3-642-12275-0\_31}
}