Reviewer Instructions
Introduction
Thank you for agreeing to review papers for NLDL, which aims to showcase high-quality research in the field of machine learning. We value your expertise and commitment to ensuring the quality of the papers submitted. In this guideline, we outline key considerations and criteria for evaluating submissions.
General Review Criteria
Coherence and Correctness: The primary criterion for acceptance is the clarity and correctness of the paper. Ensure that the paper is logically structured and that the presented methodology, experiments, and results are sound. Identify any errors or inconsistencies, and provide constructive feedback for improvement.
Incremental Contributions: Recognize that not all papers need to introduce groundbreaking innovations. Incremental advances in existing knowledge are valuable contributions to the field. Assess whether the paper adds meaningful insights, even if they build upon prior work. In this sense, incremental work is not a basis for rejection.
Application Papers: We welcome papers that focus on real-world applications of machine learning. However, such papers must meet the following additional criteria:
a. Non-Trivial Challenges: The paper should clearly articulate the challenges faced in the application domain and explain why solving these challenges is not trivial. Evaluate the extent to which the authors demonstrate a deep understanding of the specific domain problems.
b. Solution Description: Evaluate the proposed solution's novelty and effectiveness in addressing the identified challenges. Assess whether the solution has practical relevance and can be applied beyond the specific case studied.Reproducibility: Assess the paper's reproducibility by evaluating whether the authors provide sufficient information, code, and data to replicate their experiments. Highlight any deficiencies in reproducibility and suggest improvements.
Clarity and Presentation: Evaluate the clarity and organization of the paper. Ensure that the writing is clear, concise, and well-structured. Identify areas where the authors can improve explanations, figures, and tables to enhance the paper's comprehensibility.
Ethical Considerations: Check for ethical considerations related to data collection, use, and potential societal impacts. If ethical concerns are identified, make sure they are addressed or discussed appropriately in the paper.
Specific Review Guidelines
Novelty: While we accept incremental papers, they should still contribute something to the field (either an empirical or theoretical insights). Assess the extent to which the paper advances the state of the art or provides valuable insights.
Experimental Rigor: Evaluate the experimental design, methodology, and statistical analysis. Ensure that experiments are well-designed, reproducible, and draw meaningful conclusions.
Related Work: Check if the paper adequately discusses and contextualizes related work. Ensure that the authors acknowledge prior research and explain how their work differs or builds upon it.
Discussion of Limitations: Assess whether the paper openly discusses its limitations. Encourage authors to address shortcomings and provide suggestions for future work to mitigate these limitations.
Significance: Consider the significance of the paper's contributions to the machine learning community and potential practical applications. Assess whether the work has broader implications beyond the specific problem domain.
Citation and Attribution: Ensure that authors properly attribute prior work, data sources, and collaborators. Check for any potential conflicts of interest or undisclosed affiliations.
Reviewer's Summary and Recommendations
In your review, please provide a concise summary of your evaluation and specific recommendations for the paper. Summarize the strengths and weaknesses, and justify your final recommendation. Constructive feedback is essential to help authors improve their work.
Remember that our primary goal is to maintain a high standard of quality and rigor in machine learning research while being open to both novel contributions and practical applications. Your expertise and thorough evaluation are invaluable in achieving this goal.
Thank you for your dedication to advancing the field of machine learning.
Resources for reviewing
Daniel Dennet, Criticising with Kindness.
Views from multiple reviewers: Last minute reviewing advice
FAQ
Q. What is the LLM Policy for referees?
A. Referees may use any device, including an LLM, to polish their review wording, but must vouch for, and be responsible for, the accuracy of the review. It is a significant act of referee misconduct to allow an LLM to see a submission. PCs interpret showing a submission to an LLM as a deliberate referee violation of confidentiality. The PCs reserve the right to report reviewer misconduct to future computer vision conferences. These conferences then may take actions, e.g., there was a recent PAMI-TC vote that CVPR reviewer misconduct may lead to a 2-year submission ban.