[Guidelines] [Review Home] [List submissions] [Preferences] [Change Password] [Documentation]

Review Guidelines for TEST 2007

Schedule

XXX to YYY: Individual review.
Reviewers do not communicate with each other about the submissions. Please enter all the assigned reviews by the deadline.

XXX to YYY: Discussion phase.
Reviewers can see each other's reviews and discuss them using the review web site. Hopefully, we can decide (almost) everything by YYY.

XXX: Program-Committee meeting.

XXX to YYY: Wrap up.
Time to check the comments again before they are sent to the authors.

Assignments

Most submissions are assigned to three reviewers (except PC-member papers that are assigned to more reviewers). Of course, you are allowed (and encouraged) to review papers that were not assigned to you, but please do so only after you reviewed your assigned lot. The chair(s) may also ask you to review additional submissions if the need arises.

External reviewers

You are encouraged to ask others outside the committee to help evaluate the papers. External readers should apply the usual confidentiality rules for conference submissions, in particular, not to distribute the paper to other people. It may be necessary to adjust an outside referee's score to make it compatible with your own system. You should understand the paper and external readers' comments well enough that you can discuss them in the public discussions phase. Please mark the names of the external reviewers in the subreferee box on the review form. After the final decision, the chair(s) will send out a list of all the sub-reviewers so you can check that none of your sub-reviewers were forgotten.

Do not forward your account name and password to a subreferee. The review site contains very sensitive information. The program committee discussions should remain confidential within the program committee.

Writing a Report

Each reviewer assigns a grade (1-6) to each reviewed paper to reflect the recommendation on acceptance/rejection for the paper, as well as a weight (1-3) to reflect the confidence of the reviewer in the recommendation. Grades for other specific criteria may also be entered. Please use the scale below when completing the review form.
Overall grades:
1: Very weak submission, would be an embarrassment to accept
2: Weak submission, should be rejected
3: Rather neutral, but I lean toward rejection
4: Rather neutral, but I lean toward acceptance
5: Solid submission, should be accepted
6: Strong submission, would be a shame to reject
Confidence levels:
1: An educated guess
2: Quite confident (but I did not check many of the details)
3: Confident (I know the area and I studied the paper in sufficient detail)
Technical:
1: Low
2: Adequate
3: Good
Editorial:
1: Poorly written
2: Could use some improvement
3: Well written
Suitability:
1: Does not belong in this conference
2: Marginal w.r.t. the scope of the conference
3: Fits the conference

Report contents

Reports will consist of grades, a weight, and comments. The comments should help the authors as well as other committee members understand your opinions. Please don't write one-line reviews and don't over-use committee-only comments. There is no need to copy the comments to authors in the committee-only field. The bulk of the report should be visible to the authors (including summary of contributions, general evaluation, and specific comments). Authors of rejected papers deserve to know why their papers were rejected, and we want accepted papers to be improved based on our comments.

There is no additional structure to the reports that you have to follow. Here are a few points that you may want to refer to when organizing and writing your report.

Contacting authors

If you need to contact the authors of a submission, you can do this either through the chair(s) or directly. In the latter case, please let the chair(s) know that you contacted the authors.

The Discussion Phase

Please use the web site discussion boards for the discussions (and not the e-mail list); this will ensure that people who join in the discussion later can follow its history. (Conversation on the discussion boards are not accessible to program committee members who are authors of the paper in question or otherwise have a conflict of interest.)

Have fun.

The chair(s)
[Guidelines] [Review Home] [List submissions] [Preferences] [Change Password] [Documentation]