Area Chair Instructions

Introduction

As an Area Chair for NLDL, your role is pivotal in ensuring the quality and fairness of the paper selection process. This guideline is designed to provide you with clear instructions and criteria to guide your decision-making and interactions with reviewers. Please ensure that you align your decisions with the conference's objectives and the expectations outlined in the reviewer guidelines.

General Responsibilities

1. Paper Assignment: Assign papers to reviewers with expertise in the respective areas, ensuring that each submission receives a fair and expert evaluation.

2. Expertise Match: Verify the expertise of reviewers and make adjustments if necessary to ensure that they are qualified to evaluate the assigned papers.

3. Communication: Maintain clear and timely communication with reviewers and authors, facilitating discussions as needed while preserving reviewer anonymity.

4. Quality Control: Oversee the review process to ensure that reviewers adhere to the conference's guidelines and standards.

Specific Area Chair Guidelines

Coherence and Correctness

1. Primary Criteria: Emphasize that the primary criterion for acceptance is the coherence and correctness of the papers. Encourage reviewers to thoroughly assess the clarity and logical soundness of the submissions.

2. Quality Assurance: Review reviewer comments to identify potential discrepancies or concerns related to correctness and coherence. Address any discrepancies through reviewer discussions if necessary.

Incremental Contributions

3. Clarification: Clarify that incremental contributions are acceptable and valuable but must still represent a meaningful advance in knowledge. Encourage reviewers to evaluate the significance of incremental contributions within the context of the field.

4. Contextualization: Encourage reviewers to assess how well the paper positions its work within the existing literature. Ensure that reviewers appreciate the significance of incremental advancements in knowledge.

Application Papers

5. Challenge Identification: Stress the importance of evaluating application papers based on the non-trivial challenges they address in the specific domain. Ask reviewers to carefully assess whether the challenges are clearly articulated.

6. Solution Evaluation: Ensure that reviewers critically evaluate the novelty, effectiveness, and practical relevance of the proposed solutions to address the identified challenges.

7. Discussion: Encourage reviewers to assess whether the paper effectively explains why the solutions to the application challenges are not trivial and how they contribute to the broader understanding of machine learning.

Reconciliation of Reviewer Feedback

8. Discrepancies: If reviewers provide conflicting recommendations, facilitate a discussion to reach a consensus. Engage reviewers in a constructive dialogue to address differing viewpoints and consider additional expert opinions if necessary.

9. Reviewer Feedback: Consolidate reviewer feedback, emphasizing key points and suggestions for authors to improve their submissions.

Ethical Considerations

10. Ethical Review: Remind reviewers to consider ethical aspects, such as data privacy, fairness, and potential societal impacts, and assess whether these aspects are appropriately addressed or discussed in the papers.

Recommendations to Authors

11. Clear Guidance: Provide clear, constructive guidance to authors based on the reviewer feedback. Encourage authors to address reviewer concerns, improve clarity, and enhance the quality of their submissions.

12. Consistency: Ensure that the feedback and recommendations to authors align with the conference's guidelines and criteria, emphasizing the importance of coherence, correctness, significance, and ethical considerations.

Final Decision

13. Decision Justification: Make well-informed decisions based on reviewer feedback and discussions. Justify your decisions in terms of the paper's quality, adherence to guidelines, and potential contribution to the field.

Conclusion

Your role as an Area Chair is integral to the success of our conference. By adhering to these guidelines and working closely with reviewers and authors, you will help maintain the conference's high standards of quality and fairness while promoting both incremental contributions and impactful application papers in the field of machine learning.

Thank you for your commitment to advancing the research in this domain.

FAQ

About the review process

• Q. I found that the submission is not anonymized correctly.  What should I do?
  A. Contact the Program Chairs immediately.  You can send a message through OpenReview and put the PCs as readers.  If you have not received a response through OpenReview in two days, please contact through email.

• Q. Why I cannot see the ratings in the reviews?
  A.  We are evaluating the impact of the numeric scores over the final recommendations.  In this iteration of the conference, we, the PCs, hide the scores and will be used afterwards.  You should made your decision based on the descriptions in the review and discussions.

About LLMs

• Q. What is the LLM Policy for reviewers?
  A. Reviewers may use any device, including an LLM, to polish their review wording, but must vouch for, and be responsible for, the accuracy of the review. It is a significant act of reviewer misconduct to allow an LLM to see a submission. PCs interpret showing a submission to an LLM as a deliberate reviewer violation of confidentiality. The PCs reserve the right to report reviewer misconduct to future machine learning and related conferences. These conferences then may take actions, e.g., there was a recent PAMI-TC vote that CVPR reviewer misconduct may lead to a 2-year submission ban.

• Q. How will the LLM policy be implemented?
  A. An author may complain to their AC that a summary (and/or other parts of the review) have been prepared by an LLM that has seen the paper. Such a complaint would need to be supported by an example summary (or other part of the review) prepared by the author giving the paper to an LLM. If this matches the reviewer’s comments sufficiently, ACs will pass the complaint on to PCs who are then entitled, but not required, to act. Complaints must not appear on the rebuttal, but be submitted on a separate form. PCs strongly discourage frivolous complaints. Authors should be aware that a complaint to an AC about a review prepared by an LLM without reasonable evidence in support of that complaint, is wasting the ACs time.