Relevation: A raising or lifting up. - Wiktionary
Relevation is a system for performing relevance judgements for Information Retrieval system evaluation. Documents and queries can be uploaded to the system via the web interface. Judges can browse the uploaded documents and queries and provide their relevance assessments.
Relevation is aimed at anyone wishing to collection relevance assessment for information retrieval evaluation evaluation.
Relevation! is build using Python's Django web framework and Twitter's Bootstrap.
- Checkout relevation from github with:
git clone https://github.com/ielab/relevation.git - Copy your documents into the
relevation/documentsfolder. - Start Relevation! by running
./manage.py runserver. Relevation! will be running at http://127.0.0.1:8000 - Go to Setup page and upload your queries and document pool. (Queries are in the form queryId[tab]queryText; the pool is in standard TREC results format.)
You can also look at a demo that shows how to setup Relevation! from scratch; the demo is available at https://vimeo.com/ielab/relevation.
Relevation has been used in anger for:
- CLEF eHealth Evaluation Lab: 2013-2016
- ECIR diagnose me paper
- Relevance assessment collection for Bevan Koopman's PhD thesis
Relevation was designed by Bevan Koopman and Guido Zuccon. Relevation is an open source project still in development.