SC18 enhances the usual publishing process to improve the quality of the paper selection and facilitate the author’s requirements to publish their latest results. The enhancements are based on the process used at the ACM PASC’16 conference, which is now also adopted by IPDPS’17. At SC18, we provide a light version of the principles identified at ACM PASC’16. For this, we introduce two major changes:
- Authors will have the opportunity to provide a lightweight revision during the review process (in addition to an optional rebuttal as in the previous years).
- Some papers will be given the opportunity to submit a full revision to enter a second competitive review phase.
Lightweight Revision and Rebuttal
After the first reviews are received, every paper can be resubmitted with a lightweight revision with clearly marked changes and a rebuttal. The rebuttal should be used to clarify misunderstandings and technical errors in the reviews. The lightweight revision can be used to address reviewer comments and misunderstandings directly in the draft text. The authors may also add new results in support of their claims. Note that this differs from traditional rebuttals, in that authors are encouraged to add new material to their papers to address reviewer comments. Adding directly to the draft is more convincing than stating intent to do so (as in a rebuttal).
Due to the predefined timeline of the SC18 conference, reviewers will have very little time to review the changes made in the lightweight revisions. Thus, we will require that the authors clearly mark the changes in the revised draft, or provide a summary of the changes in the rebuttal text (which remains limited to 750 words). As before, the committee cannot give any guarantee that all edits and comments are taken into account, particularly if they are not clearly documented.
Major Revision (for second review)
During the physical program committee meeting, a decision for each paper from the following categories will be made: Accept, Minor Revision, Major Revision, Reject. The minor revision is similar to the well-known shepherding process. In the standard Accept/Reject selection process, many papers with borderline evaluations have a large random component in their final decision, which leads to high variance in the final quality. To address this, SC18 introduces the major revision process. The 5-10% of the papers that are promising, but for which no clear decision can be made at the program committee meeting, will be invited to submit a major revision for a second review. The final decision will be made in an online discussion by the program committee. We do not expect that this process will change the overall acceptance rate, which is traditionally around 20%. So we expect a competitive process with accept/reject decisions for the revised manuscripts.
SC follows a two-part submission process, with abstracts due by March 19, 2018 and full papers by March 28, 2018 (No extensions). Abstracts and papers must be submitted to the online submissions system. A sample submission form is also available.
An abstract submission is required in order to submit a full paper. Again, no extensions will be granted for papers this year.
Submissions are limited to 10 pages using 10pt fonts in the IEEE format. The 10-page limit includes figures, and tables, but does not include references and appendices, for which there is no page limit. Reproducibility initiative dependencies (Artifact Description or Computational Results Analysis) are also not included in the 10-page limit.
As in previous years, submissions will be double-blind. We note that track chairs will be able to view the authors of submissions during the selection to monitor conflicts and manage the overall process. Further author instructions can be found here.
All submissions must indicate one of the nine designated tracks as the primary area of contribution. One of the remaining areas may be indicated as a secondary area of contribution.
Submission material cannot overlap substantially with any paper previously accepted for publication or under review by any conference or journal during the SC review process. Authors should follow IEEE publication policies.
Members of the Technical Papers Committee are limited to at most three submissions, regardless of area of contribution. The Area Chairs and Vice-Chairs are limited to at most one submission, not in their area. The Technical Papers Co-chairs, General Co-chairs and the Technical Program Co-chairs are not permitted to submit any papers, regardless of area of contribution.
SC18 Review Criteria
The SC18 Technical Papers Committee will rigorously review all submissions with the goal of selecting the best technical contributions across both established and emerging areas of HPC. SC18 will continue the tradition of author rebuttals of reviews and extend it with optional lightweight revisions. The review process acceptance criteria will concentrate on originality, technical soundness, presentation quality, timeliness, impact and relevance to SC. Papers that focus on the context of a single node, core, thread, or GPU should clearly establish how the work will impact supercomputing.
The SC18 Technical Papers Committee is divided into nine subcommittees (or area committees). The nine area committees correspond to the nine areas of contribution: Algorithms; Applications; Architectures and Networks; Clouds and Distributed Computing; Data Analytics, Visualization and Storage; Performance; Programming Systems; State-of-the-Practice; and System Software. This division helps to ensure that each submission is reviewed by the committee members most qualified to judge its originality, technical soundness, timeliness, impact and relevance. The area of contribution that authors select at submission time will serve as the primary mechanism to determine the subcommittee to which a submission is assigned.
The review process follows several stages. First, the Technical Papers Committee Co-chairs review each submission to ensure that it is assigned to the appropriate subcommittee. As stated above, the selected area of contribution is the primary guidance for this step. However, the Co-chairs, in consultation with the Co-chairs of each subcommittee, review abstracts to ensure that the selected area of contribution is appropriate. They also consider the relative balance across subcommittees. Thus, some papers are assigned to a subcommittee other than the one that corresponds to the selected area of contribution. In addition, any submission from an Area Committee Co-chair will be automatically assigned to a different subcommittee to avoid any appearance of undue influence.
In addition to reviewing the area of contribution to which each submission is assigned, the Technical Papers Committee Co-chairs and the Area Committee Co-chairs identify conflicts of interest (COI) for members of the Technical Papers Committee. These initial conflicts are determined based on the institutional affiliations of the authors and COI information provided by committee members.
Once submissions are closed and all papers are assigned to subcommittees, each Technical Program Committee member reviews the abstracts to all papers assigned to his or her subcommittee. Based on the abstracts, the subcommittee members indicate which submissions they are qualified to review, and of these, which they would prefer to review. This process helps to ensure that each submission is reviewed by the committee members most qualified to judge its originality, technical soundness, timeliness, impact and relevance. In addition, Technical Program Committee members review COI designations from authors to identify any COIs.
Once all committee members have completed their bids, the Area Committee Co-chairs use an automated algorithm to assign papers to committee members. Each paper is assigned to four committee members, who are expected to review the papers themselves. Committee members are expected to complete all of these “first round reviews” prior to the rebuttal period. Committee members can solicit assistance from additional experts, but they are expected to read all papers that are assigned to them and to “own” their reviews. To facilitate this requirement, SC18 employs a large overall committee that should ensure no reviewer is assigned more than ten first round reviews (the expected number of first round reviews per committee member is eight).
After the first round review period, the authors are notified that the rebuttal period is open. Authors should follow the guidance about rebuttals on the FAQ. Submissions for which the first round review scores are ambiguous require additional reviews. Several factors can lead to ambiguity. For example, ambiguity may arise from a wide variance in scores or from scores that do not strongly indicate either acceptance or rejection. SC18 solicits additional reviews of papers as needed. Each committee member is expected to provide two second round reviews on average. These reviews typically are not available for rebuttal.
Following the rebuttal period and the second review round, each area committee meets in person to discuss the final disposition of each submission. This face-to-face meeting allows committee members to clarify their reviews and to reach agreement about the originality, technical soundness, presentation quality, timeliness, impact and relevance to SC of each submission. The face-to-face meeting determines a disposition of each submission in the categories Accept, Minor Revision, Major Revision, Reject. The major revision is granted to the authors if no clear consensus was reached. Major revisions are reviewed again in an online discussion of the committee and a final decision will be made.
The minor revision is used to ensure that the authors enact specific changes that the committee deemed necessary for acceptance. The shepherd or track chair acts as the representative of the committee. If the authors are uncomfortable with the requested changes then they are permitted to withdraw the submission. Once the shepherd or track chair determines that the authors have satisfactorily implemented the changes then the paper will be accepted. Failure to enact the changes may result in rejection of the submission.
For other questions, please review the Papers FAQ.