Tiny Papers Strike Back: 2023 Reflections and 2024 Announcement
1. Philosophy: why we do this
The year 2023 will be remembered as one where unprecedented and unparalleled advances on diversity, equity and inclusion (DEI) are made, by having a major machine learning conference successfully held in the African continent, for the first time in history. According to the Closing Remark, this simple change of location (while acknowledging that changes like this are never really simple) has resulted in a more than 13-fold increase (from 16 to 261) of African-based attendees.
As DEI Chairs this year, even though the majority of work pushing the DEI frontier was done when the conference location was decided, we still carry the responsibility of furthering the status quo. In January, the chairs (Rosanne, Krystal and Tom), with the support of the General Chair, Yan Liu, after meeting weekly for a few months, launched a Tiny Papers Track, an experimental paper submission track to welcome underrepresented, and non-traditional researchers into the ICLR community. As its name suggests, the track enforces a simplified format: no more than 2 pages of main content, to encourage the sharing of early results, pilot ideas, and preliminary findings.
Like any paper submission track, this effort involves putting out a call for submission, opening a submission portal, setting up a peer review system where reviewers, area chairs (ACs), and emergency reviewers and ACs are recruited, assigned tasks and consulted for decisions.
But unlike most paper tracks, we adopt an approach that embraces the principles of affirmative action, entailing the deliberate provision of advantages and preferences to historically disadvantaged populations. To that end, we ask every submission to include at least one key author that comes from a URM background, in terms of, for example, race, gender, age, geographical location, distability, or neurodiversity status. This involves adding a required section, an URM statement, to the paper submission, with an acknowledgment sentence that can be as simple as: The authors acknowledge that at least one key author of this work meets the URM criteria of the ICLR 2023 Tiny Papers Track. This honor system of self-identification respects individuals’ privacy and is aligned with a pluralistic view of advantage, representation, and systemic bias.
2. Practice: how we did this for ICLR 2023
The initiative was brainstormed, designed, and implemented for four months, before announcing to the world on January 23, 2023, and submission portal open February 1st, and submission deadline Feb 28th.
We received 219 submissions over a wide variety of topics. 580 unique total authors were involved in making submissions, with 526 of them making 1 submission, and 54 making 2 or more submissions.
We also recruited reviewers on the same day of the announcement. In the end, a total of 173 reviewers, 42 meta-reviewers (ACs) worked with us to ensure that all paper submissions have 3 or more reviews submitted. Reviewers asked to evaluate according to the CCR: Clear, Correct, & Reproducible criteria. Instead of the traditional “accept” or “reject” decision, we also adopt a novel categorization of decisions that consist of:
- Invite to present (notable)
- Invite to present
- Invite to archive
- Invite to revise
As the names suggest, those decisions reflect whether a submission meets the quality to be presented at our Tiny Papers Showcase Day, archived, or if it requires revision. Below is the distribution of the initial decision.
For those invited to revise, after revision they are given final decisions of either “Invite to archive” or “Not invite to archive”. 34 out of 58 (58.6%) submissions were changed categories from “Invite to revise” to “Invite to archive”. In the end, 194 of 219 (88.6%) submissions were accepted to archive. All archived papers can be found at DBLP.
On May 5, 2023, the last day of ICLR 2023, we hosted Tiny Papers Showcase Day, in Kigali, Rwanda. 86 papers and their authors were invited to give an oral presentation and/or poster presentation. It was a lovely day.
3. Prospect for 2024
Post-ICLR 2023, we received lots of inquiries: “Is Tiny Papers happening again? At ICLR or elsewhere?” Today, we can finally answer affirmatively, for ICLR 2024, to be held at Vienna, Austria.
We made a few changes this year to improve the program:
- Moved deadlines much earlier. An area for improvement for Tiny Papers 2023 was that it took too long to plan and we were left with too little time to execute. With such lessons learned, we moved this year’s deadlines months earlier.
- Appointed dedicated Tiny Papers Chairs who will work with DEI chairs to ensure the success of this program.
- Shared deadlines and reviewer pool with blog post track. The blog post track has been running for 3 years and has demonstrated great success and momentum. It also has the same spirit of encouraging alternative formats of scientific publications. Therefore this year we are combining forces.
For more information, read the Call for Tiny Papers 2024. Hope to see your submissions there!
Thomas F Burns and Krystal Maughan
ICLR 2024 Tiny Papers Chairs
Rosanne Liu
ICLR 2023-2024 DEI Chair