NB — The numbers might not always work out here, there are missing data from the analyses due to conflicts.
This blog post covers outcomes from the CHI 2024 Program Committee (PC) meeting, which took place from the 16-18th January 2024.
After the first round of reviews, 1651 submissions were invited to submit revisions. External reviewers and Associate Chairs (ACs) reviewed and discussed these revised submissions asynchronously and made individual binary accept/reject decisions. Submissions were there then discussed in Subcommittees at the PC meeting where final accept/reject decisions were made.
Overall acceptance rates
The Conference Program Committee accepted 1060 submissions. The overall acceptance rate for the Papers track was 26.4%. Of those submissions that were revised and resubmitted, 64% were accepted. The overall acceptance rate for “short” submissions (< 5000 words) was 10% (54/551 submissions). The overall acceptance rates for “standard” and “excessive” submissions was 29% (1006/3458 submissions). Seventeen submissions of “excessive” length were accepted (of 66,26%).
Acceptance rates by subcommittee
Submissions to the CHI Papers track are made to one of eighteen subcommittees. We considered the submission rates to these subcommittees and their respective R+R rates in other blog posts. Final acceptance rates varied between 34% (Accessibility and Aging, Access) and 18% (User Experience and Usability, UX). Figure 1 shows the acceptance rates for each subcommittee.
Figure 1: Acceptance rates by subcommittee, CHI 2024 Papers
Recommendations
Submissions to the CHI Papers track receive recommendations from reviewers, not scores. While it may not make sense to compute and plot mean scores, we can still have a closer look at the pattern of recommendations. Do all accepted submissions have a ‘clean sweep’ of accept recommendations? What is the mix of recommendations on rejected submissions? Figure 2 shows these patterns, focusing on R+R submissions where the final recommendation from reviewers was either Accept or Reject and where there were four reviews in total (edge cases are shown below).
Figure 2: How do the recommendations of reviewers look after the PC for papers invited to R+R
Not all papers finished with four recommendations. A couple had six, fifty-nine had three. The following table comprises the recommendation counts of all R+R submissions that did not finish the process with four recommendations.
| Decision | # Accept Recommendations | # Reject Recommendations | Total # recommendations | n |
|---|---|---|---|---|
| Reject | 2 | 3 | 5 | 52 |
| Accept | 3 | 0 | 3 | 43 |
| Reject | 1 | 4 | 5 | 41 |
| Accept | 4 | 1 | 5 | 20 |
| Accept | 5 | 0 | 5 | 16 |
| Reject | 0 | 3 | 3 | 15 |
| Accept | 3 | 2 | 5 | 8 |
| Reject | 0 | 5 | 5 | 8 |
| Reject | 3 | 2 | 5 | 4 |
| Accept | 2 | 3 | 5 | 2 |
| Accept | 2 | 1 | 3 | 1 |
| Accept | 3 | 3 | 6 | 1 |
| Reject | 2 | 4 | 6 | 1 |
Bonus Chartjunk
The Program Committee meeting was considering over 1600 R+R submissions. It discussed and accepted or rejected these over the course of three days. It’s busy! Precision Conference, the tool that is used to manage the submission process, keeps an ‘action log’. When someone updates their review, it gets updated. When someone makes a new submission, it gets updated. When a decision is reached, the log is updated. Figure 3 shows how activity ramped up during the PC meeting. (We’ve stripped out the “sends email to contact author” and “send eRights to ACM” events – a lot of these happen at the same time and the chart is less interesting to look at.)
Figure 3: PCS gets busy during the PC meeting!
Datatables
Figure 1 shows the acceptance rates across the PC’s subcommittees. The underlying data is provided below.
| Subcommittee | Number of accepted submissions | Total number of submissions | Acceptance rate |
|---|---|---|---|
| Access | 101 | 300 | 34% |
| Critical | 68 | 218 | 32% |
| Devices | 39 | 142 | 28% |
| Privacy | 57 | 209 | 28% |
| Systems | 81 | 283 | 28% |
| Apps | 62 | 235 | 26% |
| Health | 78 | 318 | 24% |
| PeopleQual | 61 | 260 | 24% |
| Viz | 42 | 173 | 24% |
| Design | 65 | 295 | 22% |
| Games | 35 | 157 | 22% |
| Ibti | 42 | 198 | 22% |
| CompInt | 56 | 283 | 20% |
| IntTech | 65 | 314 | 20% |
| Learning | 53 | 268 | 20% |
| PeopleMixed | 44 | 219 | 20% |
| PeopleStat | 52 | 252 | 20% |
| UX | 59 | 327 | 18% |
Figure 2 shows the variety of recommendation permutations reached by reviewers on R+R submissions. The more esoteric permutations are given in a table, but here is the full dataset:
| Decision | # Accept Recommendations | # Reject Recommendations | Total # recommendations | n |
|---|---|---|---|---|
| Accept | 4 | 0 | 4 | 825 |
| Reject | 0 | 4 | 4 | 244 |
| Reject | 1 | 3 | 4 | 144 |
| Accept | 3 | 1 | 4 | 94 |
| Reject | 2 | 3 | 5 | 52 |
| Accept | 3 | 0 | 3 | 43 |
| Reject | 1 | 4 | 5 | 41 |
| Reject | 2 | 2 | 4 | 27 |
| Accept | 4 | 1 | 5 | 20 |
| Accept | 5 | 0 | 5 | 16 |
| Reject | 0 | 3 | 3 | 15 |
| Accept | 2 | 2 | 4 | 10 |
| Accept | 3 | 2 | 5 | 8 |
| Reject | 0 | 5 | 5 | 8 |
| Reject | 3 | 2 | 5 | 4 |
| Accept | 2 | 3 | 5 | 2 |
| Accept | 1 | 3 | 4 | 1 |
| Accept | 2 | 1 | 3 | 1 |
| Accept | 3 | 3 | 6 | 1 |
| Reject | 2 | 4 | 6 | 1 |
A full breakdown of the countries from which submissions were received (and whether one or more submission from that country/territory was accepted) to the Papers track is given below.
| Country/ Territory |
Authorships on rejected submissions | Authorships on accepted submissions | Total authorships | Proportion of authorship on accepted papers |
|---|---|---|---|---|
| United States of America | 3094 | 1958 | 5052 | 38% |
| China | 1263 | 404 | 1667 | 24% |
| Germany | 825 | 316 | 1141 | 28% |
| United Kingdom | 716 | 293 | 1009 | 30% |
| Canada | 436 | 257 | 693 | 38% |
| South Korea | 372 | 192 | 564 | 34% |
| Australia | 353 | 147 | 500 | 30% |
| Japan | 352 | 131 | 483 | 28% |
| Netherlands | 224 | 76 | 300 | 26% |
| Switzerland | 145 | 75 | 220 | 34% |
| Finland | 166 | 44 | 210 | 20% |
| Denmark | 108 | 63 | 171 | 36% |
| Taiwan | 128 | 30 | 158 | 18% |
| France | 106 | 47 | 153 | 30% |
| Singapore | 77 | 52 | 129 | 40% |
| Austria | 98 | 28 | 126 | 22% |
| Sweden | 62 | 41 | 103 | 40% |
| India | 68 | 32 | 100 | 32% |
| Portugal | 46 | 42 | 88 | 48% |
| Italy | 63 | 4 | 67 | 6% |
| Bangladesh | 52 | 6 | 58 | 10% |
| Ireland | 38 | 18 | 56 | 32% |
| Hong Kong S.A.R. | 31 | 24 | 55 | 44% |
| Spain | 28 | 25 | 53 | 48% |
| New Zealand | 38 | 12 | 50 | 24% |
| Israel | 38 | 10 | 48 | 20% |
| Belgium | 33 | 11 | 44 | 24% |
| Brazil | 34 | 8 | 42 | 20% |
| Norway | 29 | 4 | 33 | 12% |
| Luxembourg | 21 | 11 | 32 | 34% |
| Poland | 21 | 5 | 26 | 20% |
| Turkey | 17 | 5 | 22 | 22% |
| Philippines | 21 | 0 | 21 | 0% |
| Cyprus | 15 | 0 | 15 | 0% |
| Czech Republic | 15 | 0 | 15 | 0% |
| Qatar | 13 | 1 | 14 | 8% |
| Kenya | 13 | 0 | 13 | 0% |
| Iran | 10 | 2 | 12 | 16% |
| Saudi Arabia | 4 | 6 | 10 | 60% |
| Ecuador | 7 | 2 | 9 | 22% |
| Estonia | 5 | 2 | 7 | 28% |
| Pakistan | 7 | 0 | 7 | 0% |
| Macedonia | 6 | 0 | 6 | 0% |
| Malawi | 0 | 6 | 6 | 100% |
| Slovenia | 4 | 2 | 6 | 34% |
| South Africa | 6 | 0 | 6 | 0% |
| Uruguay | 6 | 0 | 6 | 0% |
| Romania | 2 | 3 | 5 | 60% |
| Malaysia | 3 | 1 | 4 | 24% |
| United Arab Emirates | 4 | 0 | 4 | 0% |
| Iceland | 3 | 0 | 3 | 0% |
| Mexico | 3 | 0 | 3 | 0% |
| Nigeria | 2 | 1 | 3 | 34% |
| Rwanda | 3 | 0 | 3 | 0% |
| Bahrain | 2 | 0 | 2 | 0% |
| Ghana | 1 | 1 | 2 | 50% |
| Macau S.A.R | 1 | 1 | 2 | 50% |
| Peru | 2 | 0 | 2 | 0% |
| Russia | 2 | 0 | 2 | 0% |
| Republic of Serbia | 1 | 1 | 2 | 50% |
| Thailand | 2 | 0 | 2 | 0% |
| Argentina | 1 | 0 | 1 | 0% |
| Armenia | 1 | 0 | 1 | 0% |
| Chile | 1 | 0 | 1 | 0% |
| Colombia | 0 | 1 | 1 | 100% |
| Costa Rica | 1 | 0 | 1 | 0% |
| Croatia | 1 | 0 | 1 | 0% |
| Egypt | 0 | 1 | 1 | 100% |
| Indonesia | 0 | 1 | 1 | 100% |
| Jordan | 1 | 0 | 1 | 0% |
| Kazakhstan | 1 | 0 | 1 | 0% |
| Namibia | 1 | 0 | 1 | 0% |
| Sri Lanka | 1 | 0 | 1 | 0% |
| United Republic of Tanzania | 0 | 1 | 1 | 100% |
| Ukraine | 1 | 0 | 1 | 0% |
| Vietnam | 1 | 0 | 1 | 0% |
CHI 2024 Sponsors
Champion
Contributing
