Adhoc Retrieval

News | Sub-tasks | Administrivia | Submissions | Task Coordinators

The FIRE ad hoc task is similar to the TREC ad hoc task. Its objective is to evaluate the effectiveness of retrieval systems in retrieving accurate and complete ranked lists of documents in response to fifty one-time information needs. The FIRE 2012 ad hoc task focuses specifically on South Asian languages. This is the fourth year of the FIRE Adhoc Task. New participants are welcome!

News

For the Bengali test collection, documents in the range 2004 September-December and 2007 January-September were left out last year. The set in its entirety is being offered this year.

We are supplementing the FIRE 2008 / 2010 Bengali Anandabazar Patrika (ABP) corpus, which had had news articles of the years spanning 2004 to 2007, with a longer span ranging from 2001 to 2010.
The BDNews 24 collection covers news in English and Bengali from 2006 - 2010.

Sub-tasks

Mono-lingual retrieval

Bengali
Gujarati
Hindi
Marathi
Tamil

Cross-lingual retrieval

Queries : Bengali / English / Gujarati / Hindi / Marathi / Tamil / Telegu
Documents : Bengali / English / Gujarati / Hindi / Marathi / Tamil

The Bengali and Hindi topics will also be transliterated and made available in Roman script. All participants are encouraged to submit runs using these queries as well.

Administrivia

Periodically check this website for announcements and updates.

Subscribe to fire-list to participate in discussions and be informed of any late-breaking announcements.

Submissions

Please click here to Submit Run

Each submission file should contain 1000 documents per topic, ranked 0-999, in the usual TREC / CLEF submission format, i.e. each line in the file should have the following fields:
<Query id> Q0 <DOCNO> <RANK> <SIMILARITY> <Run-ID>

Participants will need to submit a gzipped file containing retrieval results in the above format.

All participants are required to submit at least one run that uses only the title and description fields (no narrative) of the topics. There is no upper limit on the number of submitted runs. However, please assign a priority to each of your submissions. Runs will be included in the pooling process in order of priority.

Task Coordinators

Ayan Bandyopadhyay; bandyopadhyay.ayan@gmail.com
Dipasree Pal; dipasree_t@isical.ac.in
Sauparna Palchowdhury; sauparna.palchowdhury@gmail.com