OLAC Record

Title:LongEval Click-Model Relevance Judgements (Qrels)
Bibliographic Citation:http://hdl.handle.net/11234/1-5151
Creator:Galuščáková, Petra
Devaud, Romain
Gonzalez-Saez, Gabriela
Mulhem, Philippe
Goeuriot, Lorraine
Piroi, Florina
Popel, Martin
Date (W3CDTF):2023-05-22T16:24:01Z
Date Available:2023-05-22T16:24:01Z
Description:The collection comprises the relevance judgments used in the 2023 LongEval Information Retrieval Lab (https://clef-longeval.github.io/), organized at CLEF. It consists of three sets of relevance judgments: 1) Relevance judgments for the heldout queries from the LongEval Train Collection (http://hdl.handle.net/11234/1-5010). 2) Relevance judgments for the short-term persistence (sub-task A) queries from the LongEval Test Collection (http://hdl.handle.net/11234/1-5139). 3) Relevance judgments for the long-term persistence (sub-task B) queries from the LongEval Test Collection (http://hdl.handle.net/11234/1-5139). These judgments were provided by the Qwant search engine (https://www.qwant.com) and were generated using a click model. The click model output was based on the clicks of Qwant's users, but it mitigates noise from raw user clicks caused by positional bias and also better safeguards users' privacy. Consequently, it can serve as a reliable soft relevance estimate for evaluating and training models. The collection includes a total of 1,420 judgments for the heldout queries, with 74 considered highly relevant and 326 deemed relevant. For the short-term sub-task queries, there are 12,217 judgments, including 762 highly relevant and 2,608 relevant ones. As for the long-term sub-task queries, there are 13,467 judgments, with 936 being highly relevant and 2,899 relevant.
Identifier (URI):http://hdl.handle.net/11234/1-5151
Language (ISO639):fra
Publisher:Université Grenoble Alpes
Rights:Qwant LongEval Attribution-NonCommercial-ShareAlike License
Subject:information retrieval
automatic evaluation
Type (DCMI):Text
Type (OLAC):primary_text


Archive:  LINDAT/CLARIAH-CZ digital library at the Institute of Formal and Applied Linguistics (ÚFAL), Faculty of Mathematics and Physics, Charles University
Description:  http://www.language-archives.org/archive/lindat.mff.cuni.cz
GetRecord:  OAI-PMH request for OLAC format
GetRecord:  Pre-generated XML file

OAI Info

OaiIdentifier:  oai:lindat.mff.cuni.cz:11234/1-5151
DateStamp:  2023-05-22
GetRecord:  OAI-PMH request for simple DC format

Search Info

Citation: Galuščáková, Petra; Devaud, Romain; Gonzalez-Saez, Gabriela; Mulhem, Philippe; Goeuriot, Lorraine; Piroi, Florina; Popel, Martin. 2023. Université Grenoble Alpes.
Terms: area_Europe country_FR country_GB dcmi_Text iso639_eng iso639_fra olac_primary_text

Up-to-date as of: Thu Oct 5 0:43:36 EDT 2023