- How are the overall leaderboard rankings done across tasks?
- How will the results for the secret tasks be reported, if at all?
- Will there be any live feedback regarding submissions, for example, will we be notified if there are any errors in our submission or when the submission results are available?
- Currently, we plan to do no overall leaderboard ranking. There will be one ranking per task.
- Each secret task has measures associated with it, that will be published. Also, some high level metadata about the task (e.g. the domain, the size, etc.) will be shared.
- If we have any problems running the submission, we will reach out the submitters directly over email.
The first leaderboard will be up in two weeks. Please see recent updates about rolling leaderboard submissions