Each piece of description in the validation.csv summarizes a scientific paper in the candidate.csv. For each piece of description, the participants need to choose 3 best-matching papers. Please note that these predictions are ordered by their probability, from high to low.
Please refer to sample_submission.tsv for submission format. The submission files are in tsv format without any header:
description_id, paper_id (1), paper_id (2), paper_id (3)
with each line of the file corresponds to one piece of description (description_id), followed by 3 most relative papers (paper_id). These three predictions in one line shall be different from each other. Halfwidth comma shall be used between press_id and paper_id.
Please note that tsv file is separated by tab. If you use Python Pandas, you can use the parameter: sep='\t' when saving files.
Submissions are evaluated according to the Mean Average Precision @3 (MAP@3)
Where |U| is the number of press_id in the test set, P(k) is the precision at cutoff k, n is the number of predicted papers.
Sponsor：AMiner · Microsoft · biendata